Web client devs - best way to do low level Autonomi REST APIs

Development
, , ,
1

This is a topic for discussing what REST APIs for Autonomi are wanted, feedback on what exists and so on.

This is not just for dweb, although that’s what I will be focusing on. Any RESTful API is welcome here and if we can work towards common standards that would be great - but for now I just want to get some useful APIs in place so folk can try out building dynamic web apps for Autonomi.

Here’s my first question, but devs feel free to open discussion on any REST API for Autonomi.

dweb

I’m working on REST APIs for Autonomi functions and have POST/GET for a few types working (archives POST/GET, files multipart-POST/GET, data GET etc).

EDIT: For the moment I’m focused on web clients so expect JSON payloads and responses (although for /data which gets data using a datamap or data address the response is binary so you can load directly into a browser). If you need binary POST/GET let me know. I’ll be doing it anyway at some point but will prioritise based on what folk want.

So, with a JS web app client in mind here’s a simple example I made (literally yesterday):

/chunk POST/GET

I have just implemented Chunk POST/GET but to start with have the client supplying the content with a simple JSON object, which mirrors the following DwebChunk struct in Rust:

pub struct DwebChunk {
    content: String,
}

So in JSON the client would create a JSON object to POST to /chunk which looks like this:

{
  "content": "hi, I'm a chunk"
}

The response is a PutResult (in Rust) which for the above returns the following JSON to the client:

{
  "dweb_type": "Chunk",
  "status": 200,
  "status_message": "success",
  "cost_in_attos": "3",
  "file_name": "",
  "full_path": "",
  "data_address": "a31f27622645bb1a467d0e1d8e9b169de1a907f170291e1cbe28ad3c3fbfdc2c",
  "data_map": ""
}

I return similar JSON for other POST and PUT operations with content depending on the type of data being stored, and whether it succeeds.

Feedback

Comments, suggestions and requests on any of this are welcome but for now I’m wondering about the JSON used to POST a Chunk. Using a String is limited, but JSON support for binary also not great so any ideas?

What additional JSON options would be useful for the data and using what encoding? For binary it might be a base64 or IntelHex ecoding for example, both of which are discussed on StackOverflow here.

For now I will publish as is, but will update this depending on feedback. If you have requests or suggestions relating to any Autonomi data types, please reply here too.

Next I’m working on Pointer, Scratchpad and probably Vault but for now from a web JS client perspective. So if you need non-JS stuff shout!

Thanks, and if you want to try the dweb REST APIs, see here.

8 Likes
2

A few thoughts I have:

  • use HTTP status code to reflect the status, rather than embedding it in the payload
  • post binary data directly instead of embedding it in json. REST as HTTP POST is fine for this
  • the reverse as HTTP GET returning binary is good too and browsers expect it.
  • Imo, it’s ok to have a mixture of JSON endpoints and those that request or respond with binary payloads.
4 Likes
3

Very helpful comments @Traktion, thanks for taking time.

Where APIs need to return something like a datamap or address I use a standard struct/JSON object (e.g. PutResult) even if there’s an error - so you spotted a BUG there. So I will extract that and return an error response if the put fails, thanks! :folded_hands:

Indeed, /data GET (the first one I wrote) returns binary. So you can use that in the browser address bar with a datamap or data address to view content.

I’m using JSON for more complex responses such as the result of a POST/PUT. I’m still not sure if what I’m doing is the best though so hints and suggestions around these are very welcome. Being my first REST API I really appreciate your input.

Good to know :+1:, that’s what I plan but am fumbling around in the dark a bit.

For now I’m mostly focused on web JS clients, so JS dominates the APIs for complex types. I do use binary for /data GET (not sure how to do binary PUT yet, but I do have multipart form upload of file/directory in the meantime).

Where a browser JS client needs to post binary data I’m still open to feedback (as per the OP), so let me know if you have any opinion on that. For multi-part forms (local file upload) it is handled by the browser so no problem, but what if a client wants to save binary data (e.g. to a chunk or as self-encrypted data)? I’m assuming they would create a JSON object, but maybe that’s not the only option there.

Over time I hope we can come up with a common set of APIs to cater for different needs so it is important to me to incorporate your ideas and requests, adapt to a common view etc. Thanks again. :folded_hands:

4 Likes
4

I think you should also consider using custom headers for metadata, then you can have pure payload in binary/json.

5 Likes
5

I didn’t even know you could do that!

Could you help me understand what you have in mind, maybe take one API and suggest what headers etc?

Or point me to any example(s) of good practice.

I’m not sure of the pros and cons of the different approaches for different use cases. And for dweb this may be atypical because it is designed to be a local server on the same client rather than a remote.

I’m such a novice at this any good advice is invaluable. :folded_hands:

1 Like
6

One advantage of the headers approach is that you can use them in any tools that understand http – cache, load balancer etc. Your chunk example could look like this in javascript:

const response = await fetch("http://localhost:12345/chunk", {
  method: "POST",
  body: new Blob(["hi, I'm a chunk"], {type: "text/plain"}),
});

console.log(response.status); // 200
console.log(response.statusText); // OK
console.log(response.headers.get("DWeb-Attos-Cost")); // 3
console.log(response.headers.get("DWeb-Type")); // "Chunk"
console.log(response.headers.get("Content-Length")); // 32
console.log(response.headers.get("Content-Type")); // "application/octet-stream"
console.log(await response.bytes()); // Uint8Array [ 163, 31, 39, ...
3 Likes
7

I think the downside is that they are more opaque than a URL and a body though. Not a huge issue, but worth considering.

2 Likes