AntTP - Serving Autonomi data over HTTP

v0.6.0 has now been released and it is a bit of a whopper!

  • Adds pointer REST API with CRU operations.
  • Adds register REST API with CRUH operations.
  • Adds associated app_private_key CLI option to derive sub-keys and ownership.
  • Adds the concept of bookmarks as a CLI option, which is a basic pet names resolver (more to come!)
  • Adds disable uploads as a CLI option for those who want read only hosting.
  • Refactoring to move HTTP request logic to ‘controller’ and service layer code to ‘service’.
  • Renamed xor_helper to resolver_service and added support for resolving bookmarks, pointers and registers for public archives.
  • Refactoring to move more code into separate crates
  • Adds mutable caching for short period in HTML response (60s default).
  • Refactorings for resolver_service.rs to use struct for result and associated improvements.
  • Reduced info logging to remove noise.
  • Adds config to set mutable caching TTL
  • Replaced REST API POST/PUT with async calls, returning derived successful values immediately.

Some of the features are general improvements, but many were dependencies for IMIM, so I needed to get them in place ASAP.

The REST API may be subject to interface changes, as it was pieced together pretty quickly. However, it does provide a mechanism to perform POSTs/PUTs (create/update) operations asynchronously, with a derived successful value returned immediately. This hides much of the latency which can be involved and allows clients to assume success. A subsequent GET operation will hit the network and can be used to discover if the operations were successful.

I want to do a bit more to indicate this is what has happened, probably by adding a response header, but it’s a bit WIP and I’ll probably refine as I stress test it with IMIM.

Note that the async tasks will soon be added to a job processor, which will have the potential to block the network operation, until a second application has signed the task. This will provide a 2-factor style interface, to avoid bad apps from storing unexpected data. Ofc, the signing app could automatically sign in some cases to improve UX (e.g. allow batches of X actions to be allowed, allow all from app Y, etc).

There is now also a flag to turn off uploads altogether for public proxies (which was a feature request).

For REST API GETs, I have added mutable data caching, which has a 60s default TTL (can be overridden in a CLI argument though). This results in subsequent GETs retrieving the data from memory until it expires. Likewise, this 60s TTL is also used for HTTP caching headers to encourage the browser to re-use the same data for up to 60s.

While the cache is short lived, it makes a big difference to page load times, as the same API data may be retrieved many times.

Note that this new caching complements eTag caching, which is still a huge help. If the content of a mutable target is unchanged, once the target has been retrieved again, the target data will not need to be downloaded again - the browser still knows its the same content!

Moreover, the HTTP requests for archives and chunks benefit from pointer and register integration. AntTP will now find the pointer target or the latest register value, resolve it, then download the files/data as before. This applies to all file types, including videos/music/text/images, etc.

After the above, I thought it would be a shame if we then just had even longer URLs to share… so I created a basic pet names / bookmarks system too. By default, I’ve just thrown in ‘imim’ and ‘traktion-blog’, which resolve to a pointer and register respectively, which in turn resolve to data/files on the network.

What does that mean? Well, assuming you have your browser proxy setup, you can just got to this URL to see my blog! :sweat_smile:

http://imim/blog/traktion-blog

Articles can be linked as follows too:

http://imim/blog/traktion-blog/article/dog-food-is-delicious.md#article

All a bit more readable, right?

I want to expand on this ‘bookmarks’ concept to accept addresses on the network for these, which then allows a more genuine ‘pet names’ experience. For now, you can specify your own key/value pairs from the CLI.

What else does this mean for IMIM? I’ll leave that for an IMIM update post… :smiley:

Github: Release v0.6.0 · traktion/AntTP · GitHub
Dockerhub: https://hub.docker.com/r/traktion/anttp
Cargo: cargo install anttp

(P.S. I had a dependency issue with the aarch64 build, so I’ve just dropped it from the release builds for now while I investigate the cause. The old aarch64 versions will still be fine though.)

13 Likes