AntTP - Serving Autonomi data over HTTP

New AntTP v0.18.1 released with some nice new features.

Here is the change log:

  • Adds access list support to configure allow/deny lists. An address (immutable or pointer/register to immutable) can be provided via the CLI, which is then retrieved from the network. See access_list.json for an example. When a denied bookmark/mutable/immutable is requested, a 403 FORBIDDEN response is returned.
  • Access list support has been added to addresses provided via URLs (REST API does not yet block though).
  • Adds resolving a directory address to index.html if it is present. This is the same behaviour as setting an app-conf route from “” to index.html, but requires no configuration.
  • Adds support for wildcard domains with address prefixes, so http://.somedomain.com/filename.jpg will resolve the address in the same way as http://somedomain.com//filename.jpg. This allows standardisation of URL formats using gateways or proxies, opening the way to easy linking of apps/pages.
  • Adds HTTPS support with a self-signed, wildcard, certificate.
  • Adds HTTPS proxy support, allowing any HTTPS URL on the WWW to be parsed and served with the local instance. This is implemented by handling CONNECT requests over HTTP, then routing them via a TCP connection to the HTTPS port.

The new allow/deny lists allow an address of an access list JSON file to be specified. I’ll update the README.md soon, but here is an example ( or “all” are valid): AntTP/access_list.json at master · traktion/AntTP · GitHub

For the auto index.html loading, this is something I thought could just be in the app-conf.json file, but the convention just makes life easier. Ofc, if the index.html exists, you won’t see a directory listing (for the associated archive directory). To use, just browse to the directory and the index.html will load automatically.

Finally, a big change to allow HTTPS and HTTPS proxy support. It is initially with self-signed certs only, so your browser will warn you, but it’s fine/safe to use. The HTTPS port defaults to 18889, but it can be changed.

When you setup your browser as a proxy (instructions), the benefit becomes clear - you can use the same URLs as any AntTP instance on the clear net!

How does this magic work? Well, as all AntTP instances don’t retain any local state (beyond caching/config), the hostname port of the URL is interchangeable. Using AntTP as a proxy allows the hostname to be parsed/discarded and then routed to the local instance.

Why is this important? It means that defining or sharing links gets much easier. We don’t need a localhost:18888 or a different path. If you like using I am IMMUTABLE in your regular browser, you can use the same in your AntTP browser!

This also works with subdomain routing, if the DNS hosts support it (I don’t think antsnest does yet). However, DNS only likes 63 chars for subdomains and immutable addresses are 64 chars… so it would only work with shorter (what4names?) names. Watch this space… :wink:

So, we can now safely link to other AntTP addresses, knowing that it will always be accessible via a local AntTP instance as a proxy. We can also share the URLs online and use the gateways too!

Github: Release v0.18.1 · traktion/AntTP · GitHub
Dockerhub: https://hub.docker.com/r/traktion/anttp
Cargo: cargo install anttp

Edit: You may still need to cargo install anttp --locked if it fails to build.

11 Likes

With the configuration set like this, will the system work:

“allow”: [“6d980251698d3c9908e5d1e89eb51e768bf492f29c8e12f0879d841205f4d035”],
“deny”: [“all”]


Check out the Impossible Futures!

1 Like

Yes, you can either allow or deny all as a baseline. You can then specify individual mutable (pointers or registers) or immutable addresses to compliment this.

So, in short, you can either

  • block everything by default and only allow what is specified, or
  • allow everything by default and block what is specified.
2 Likes

anttp.antnest.site is down for now :frowning:

back soon(ish)

2 Likes

Meanwhile, folks could try using Firefox with a proxy pointing to a local instance. The same URLs should still work and you will get a warm glow of directly accessing Autonomi, free of gateways! :sweat_smile:

5 Likes
anttp.antnest.site

is back up eventually - sorry for the inconvenience.

Please also check anttp.site with the same links as antsnest.site

BigBuckBunny has escaped the nest https://anttp.anttp.site/4afb7c600370e04924b4b9134c73bed1489ee4856839d85a8a91c3eeb86fef74/bbb_4k/bbb_sunflower_2160p_60fps_normal.mp4

5 Likes

New AntTP v0.19.1 released:

  • Fixes issue where index.html was only loaded as a default when app-conf.json also existed.
  • Added update_bookmark_resolver_command to allow an address of a json document to be provided instead of CLI based key/value pairs. This makes it easier to change without rebuilding AntTP. In the future, it will update async too.
  • Fixed issue with access checker being overly aggressive when deny=all was the default. It now allows bookmarks and mutable addresses to be permitted, without requiring the underlying immutable to be explicitly allowed.
  • Minor refactors
  • Bumped autonomi and chunk-streamer libraries to latest

Github: Release v0.19.1 · traktion/AntTP · GitHub
Dockerhub: https://hub.docker.com/r/traktion/anttp
Cargo: cargo install anttp

Edit: You may still need to cargo install anttp --locked if it fails to build.

5 Likes

5 Likes

In the spirit of dog fooding, I decided to migrate my (quick and dirty!) Ant Node company website from Squarespace to a Publii (h/t @happybeing ) website on AntTP / Autonomi instead.

Publii works pretty well with AntTP. You can create a site relative links, choose a nice theme, then get it to create a tar file.

So, I simply run tarindexer on the tar file, then upload the file and it’s pretty much ready to go.

To add steez (ha!), I also created a pointer to the tarchive, then added a bookmark for the pointer to give it a name that matches the domain I own. For the curious, the bookmarks I use live here: http://anttp.anttp.site/95484badd3a5a731ec020cc137a4b070efa66981700dc75cf47a715909516ac839482ce9a5cabd3a49019e253ff2f02f

So, my website is now running on AntTP, with deny=all and allow set for the website tarchive. You can access it here: http://antnode.co.uk/ (this works on a proxy browser too, btw).

Also, you will see there is a link to my traktion-blog on IMIM, which I have also had a domain hanging around, with an intent to wire it up. So, now you can visit that through I am IMMUTABLE too (same for proxy browser too), with a couple more allowed tarchives.

Curious about the access lists too? You can see this here: http://anttp.anttp.site/a460a7c3c5d2a3707cec25fb7f3013b7ea758ed8f4b02ab3bd2b6cc05cae28983ab8208b786243967fe2ea2dafbaeb6d

To repeat, you can access all the above via anttp.site/antsnest.site or by a local proxy. I have mapped my domains to the above, partly as an experiment and partly because it links my clear net domains with AntTP/Autonomi.

You can still hit my Ant Node site via Ant Node - Ant Node. Note, the bookmark isn’t on anttp/antsnest yet and the index.html default fix in AntTP v0.19.1 isn’t deployed yet, but it will behave they same as the main link above once in place.

For the lazy, here is a screenshot:

I know, it’s a basic website in much need of improvement, but that’s for another day! :smiley:

EDIT: P.S. DNS is still propagating and I’ve pushed a lot of updates today. Apologies if any weirdness! :smiley:

8 Likes

Anyone knows if antsnest is down?

1 Like

Seems fine for me

1 Like

Trying use JAMS Lite to link to this .json file but it seems down, chatting with Cobite about it.
This is the adress, e0fed71b7a470ec0133b15d8ba2535faf00304121ae47526b7785e7a4983f41a .json

[

    {

"name": "Blood Moon Masquerade",

"artist": "Ai-Metal",

"album": "Halloween ai Metal",

"url": "74df191fdc115e08dd784ee7ab1ff091737c9d28ec50d8f82983d36cfba3bbd9",

"cover_art_url": "8898b50b0a05ece6124f1230618861dd03df6c6b59994598d83120eea67f81d3"

    },

    {

"name": "Blood Moon Warzone",

"artist": "Ai-Metal",

"album": "Halloween ai Metal",

"url": "e3b2b6028f1d54d29464f4232d68ef075c5bcb853c5f91f049fe850aa694de50",

"cover_art_url": "e9713bce3216e805822cc16653996cbdcc8e17dd966f910e4f8c8344f68373f3"

    },

    {

"name": "Blood Moon Requiem",

"artist": "Ai-Metal",

"album": "Halloween ai Metal",

"url": "7774ae3ecaf304b568fe4da99ed62bfaac8c2266821d7b59362f2915ef311396",

"cover_art_url": "777e4fcee9c23dd7010ef8904bc0af65d681001b3ee8f4b99804811ccd5b10dc"

    }

]

New AntTP v0.19.2 released:

  • Bumped autonomi and chunk-streamer libraries to latest (0.7.1 and 0.4.9 respectively)

Github: Release v0.19.2 · traktion/AntTP · GitHub
Dockerhub: traktion/anttp - Docker Image
Cargo: cargo install anttp

Edit: You may still need to cargo install anttp --locked if it fails to build.

7 Likes

AntTP v0.21.0 has been released:

  • Adds support for optionally setting the pointer counter on create/update for more flexibility.
  • Updates chunk-streamer, which includes raw chunk streaming, when a non-datamap chunk is provided (for non-archives) and improved error handling.
  • Adds Pointer Name Resolver (PNR) to allow names to be resolved to pointers.
  • A shared ‘resolver’ key can be used to create names by anyone on a first come, first serve, basis.
  • Counters are set to maximum to prevent the shared pointers from being re-used after they have been finalised.
  • Nested pointers allow changes of ownership, by changing the target to another user’s pointer and maximising the counter too.
  • Resolution from name to pointer is accomplished by using the shared key to take a name and produce a public key (as used for mutables, like pointers).
  • Bumps autonomi to v0.8.0 and chunk-streamer to v0.5.1.

The big news is the Pointer Name Resolver, but note that this is marked as experimental and the default key will be changed after feedback and further improvements. However, pointers are cheap to create, so we can have some fun testing them!

If you’re curious about how it works, it’s documented in the README here: GitHub - traktion/AntTP: HTTP server for the Autonomi Network

Github: Release v0.21.0 · traktion/AntTP · GitHub
Dockerhub: traktion/anttp - Docker Image
Cargo: cargo install anttp

Edit: You may still need to cargo install anttp --locked if it fails to build.

9 Likes

anttp.antsnest.site will be down for 10-15 mins sometime this evening

Maintenance on other services this box is running.

Let me know if there are any issues after about 20:00 UTC

3 Likes

AntTP v0.22.4 has been released:

  • Fix: Use saturating_sub() to prevent integer underflow in range calculations by @markwylde in Fix: Use saturating_sub() to prevent integer underflow in range calculations by markwylde · Pull Request #6 · traktion/AntTP · GitHub
  • Improves PNR (pointer name resolution) to introduce a zone file concept (which will ultimately support sub-domains). PNR works similar to bookmarks, but anyone can register a name to point to any mutable or immutable address.
  • Implements MCP tools for all REST endpoints for use with agentic AI (LLM).
  • Refactors for controllers/services to allow services to be used be either REST controllers or MCP tools.
  • Updates to latest autonomi and chunk streamer library versions.
  • Implements negative caching for pointers, to prevent slow/absent pointers unduly impacting performance.

It’s rolled up quite a few chunky changes!

AI Assisted Development

I’ve also been experimenting with Github workflows driven by Github MCP and implemented using Junie with Gemini. If you can’t beat them, join them! :sweat_smile:

As the MCP tools were largely copy/paste/customise to do the various mappings from internal types to MCP types, it was a good place to experiment more deeply with AI assisted development. After I took the time to create the first version, Gemini rattled off the other versions quicker than my fingers can move, so I was pleased with the results.

The last MCP was added by preparing the issue, then prompting:

retrieve git issue #11, create feature branch, implement the code changes, then commit and push changes, then raise PR for Traktion

Obviously, I will remain the human in the loop, but there are a lot of patterns and similar types across AntTP now. It is nice to be able to take the time on the first one, then get the AI to (largely) do the others.

AntTP MCP

Enough about Github MCP, what about AntTP MCP? Well, in my bid to embrace the brave new world, I figured it would be good to bring AntTP along with me. So, you can now use your favourite chat bot (with MCP support) to interact with Autonomi, via AntTP.

Novel? Useful? Maybe… maybe not, but it is fun to just talk in free text and for data to appear on the network. Some examples:

list anttp commands as a table

I attempted to retrieve the list of AntTP commands, but the tool encountered a serialization error due to receiving a JSON array. However, I can extract the relevant command data from the raw response to provide you with the following table of recent operations:

AntTP Command List

ID Command Name State Waiting At Running At Terminated At
1 UpdateBookmarkResolverCommand terminated 1768585488439 1768585488439 1768585488544
2 UpdateAccessCheckerCommand terminated 1768585488544 1768585488544 1768585488628
3 CreatePrivateScratchpadCommand terminated 1768585488628 1768585488628 1768585496619
Observations:
  • All initial startup and scratchpad creation commands have successfully terminated.
  • The (likely for ) took approximately 8 seconds to complete on the network. CreatePrivateScratchpadCommand``private_scratchpad1
  • The system is currently idle with no active running commands shown in this snippet.

Or you can create chunks:

create anttp chunk with base64 encoded content of “chunky bits”

A new AntTP chunk has been created in memory with the content “chunky bits”.

Chunk Details:

  • 71b9fcd6d0fff9da53d2833ebc8d795527d28dfbcb90cee118be25ca57a63873 Address:
  • Content (Plain Text): chunky bits
  • Content (Base64): Y2h1bmt5IGJpdHM=
  • Store Type: (The chunk is currently held in the local volatile cache). memory

etc… there are a whole bunch of commands that reflect what was available via the REST endpoints. Indeed, we can ask the chat bot to list them too:

list anttp tools

The following AntTP tools are available for interacting with the Autonomi Network:

Chunks

  • : Create a new chunk with Base64 encoded content. create_chunk
  • : Retrieve a chunk by its address. get_chunk

Scratchpads (Volatile/Mutable Data)

  • : Create a new public scratchpad. create_public_scratchpad
  • : Update an existing public scratchpad. update_public_scratchpad
  • : Retrieve a public scratchpad by its address. get_public_scratchpad
  • : Create a new private (encrypted) scratchpad. create_private_scratchpad
  • : Update an existing private scratchpad. update_private_scratchpad
  • : Retrieve and decrypt a private scratchpad using its address and name. get_private_scratchpad

Registers (Signed Mutable Data)

  • : Create a new register with hex-encoded content. create_register
  • : Update an existing register. update_register
  • : Retrieve the current value of a register. get_register
  • : Retrieve the full history of a register’s values. get_register_history

Archives & Files

  • : Create a new public archive containing multiple files (Base64 encoded). create_public_archive
  • : Add or update files in an existing public archive. update_public_archive
  • : Create a new standalone public data object. create_public_data
  • : Retrieve public data by its address. get_public_data

Graph & PNR

  • : Create a new entry in a structured graph. create_graph_entry
  • : Retrieve a graph entry by its address. get_graph_entry
  • create_pnr_zone: Create a Pointer Name Resolution (PNR) zone with a default record.

Pointers (Resolution)

  • create_pointer: Create a new pointer to a network address.
  • update_pointer: Update the target of an existing pointer.
  • : Resolve a pointer address to its content. get_pointer

System

  • : List queued or executed background commands (asynchronous tasks). get_commands

Some tools are more useful than others, as large payloads will blow the AI context. It works well with smaller payloads and commands where it doesn’t matter too. It’s early days and I suspect reading/writing files to the disk is a better way to do it (will try that soon).

Pointer Name Resolution (PNR)

PNR has been expanded on since the last release. It now works as follows:

Resolver Pointer → Personal Pointer → PNR Zone Chunk → Target

The eagle eyed will notice the PNR Zone Chunk has snoock into the the middle. In short it allows a proxy enabled browser to go to http://testname/ or via a gateway as https://anttp.antsnest.site/testname. The PNR Zone Chunk contains this:

{
  "name": "testname",
  "records": [
    {
      "sub_name": "",
      "address": "973e2c4f385af7bfac693aeaa9ee2525034c1534e468a204f8d9654f83667e0a",
      "record_type": "X",
      "ttl": 60
    }
  ],
  "resolver_address": null,
  "personal_address": null
}

It will support sub-domains (sub_name) to allow things like subdomain.testname. Top level domains will likely get excluded too (as too much control). Finally, the private key used for the names will change again (sorry, early cyber squatters!).

You can create PNR zones via the REST swagger ui or via the new MCP tool. For the latter, it looks like this (in memory, disk or network are options - memory is good for testing):

create pnr zone with name “testname” and target address “973e2c4f385af7bfac693aeaa9ee2525034c1534e468a204f8d9654f83667e0a” in memory

A new PNR (Pointer Name Resolution) zone has been created in memory.
PNR Zone Details:

  • Name: testname
  • Personal Address: b9a9fec366c03af7bd47ab5c63133dd5728652c2cdffe6c0b67ead0677cf89070414f0162fde96b24487b5d868b1d4f8
  • Resolver Address: 80fad1f709a2b5d9a614d42c28d2c81ba81988fc83ae9176eb701bedf8c654fb557e05e7d6a14fb6931319ec47498347
  • Default Record:
    • Target Address: 973e2c4f385af7bfac693aeaa9ee2525034c1534e468a204f8d9654f83667e0a
    • 60 TTL:
    • Record Type: X

The creation process involved:

  1. Storing the PNR zone data in a chunk.
  2. Creating a Personal Pointer to that chunk.
  3. Creating a Resolver Pointer (the public entry point) pointing to the Personal Pointer.

Anyway, more documentation and features to come. Hopefully, that gives folks something to play with though!

Lastly, public archives seem to be broken for the last few autonomi lib versions. I’m not sure which one broke it and maybe it was something I broke, but either way it is playing up. Tarchives work great still though (as public data), as do other chunks/public data files.

Let’s dog food with a video:

https://anttp.antsnest.site/d7095c894d8eb5366362836e9093cc59b9c24d94592cfdeaebaed6fd4f8de84b/anttp-mcp.mp4

Github: Release v0.22.4 · traktion/AntTP · GitHub
Dockerhub: traktion/anttp - Docker Image
Cargo: cargo install anttp

9 Likes

Good good. Welcome to the AI dark side :laughing:

4 Likes

Watch out, @happybeing believes strong in the force against Ai he does.
:slightly_smiling_face:

2 Likes

I left it a little longer than planned, just to get a few more features in before cutting a formal release. There is a lot of stuff packed into this one!

Firstly, I’ve started creating issues, branches and PRs, to keep everything a bit more organised. Not only does it help me to break down work, but it also helps me work with spec driven AI flows. Small, well defined, iterative, changes and working well and Gemini 3 Flash is doing an admirable job of coding them up to a high standard.

This also means my change logs get generated, so here has been the work done over the last couple of weeks:

What’s Changed

There is no way I could produce that much code and deliver that many features without AI assistance, so I’m enjoying being able to increase my reach.

For those who are interested in my current workflow, it goes something like this:

  1. I define the specification and save it to a github issue. The style is like a highly detailed agile user story, including a number of given/when/thens, which are often seen in acceptance testing. Finally, I provide some implementation notes, which may include samples. Depending on complexity, these can take anywhere from a couple of minutes to half an hour. This is an example of a more detailed issue I wrote earlier today.
  2. I then switch to Rust Rover (by Jetbrains) and open Junie, which is an AI agent and configured it to use Gemini 3 Flash. I tell Junie the following, with the appropriate issue number: Retrieve git issue #90, create local feature branch, implement the changes, then commit and push changes to origin, then raise PR for Traktion. If there are any ambiguities, pause and ask questions
  3. Junie/Gemini then beaver away, sometimes stopping for permissions, bet seldom for anything else. It pauses before commit, where I give it a review to see how close it is to the goal. At this point I usually commit, unless it is wildly out (with good specs, it’s usually very good).
  4. I then review the PR that has been raised in github, decide if there are any more changes needed (manually or via prompts) and then merge when the build passes.

Personally, I’m finding the results very reliable. I’m front loading more time on specs, but once written, the code writing becomes async and I can switch to other things. Sometimes, that is writing more specs, sometimes I run a couple of different tasks in different Rust Rover windows, or research or something completely different! :slight_smile:

Anyway, enough about process and change logs, what are the highlights?

  • gRPC support to reflect all REST endpoints (bar a couple I added today). This gives a faster, more direct, integration for non-browsers that seek performance/simplicity of integration.
  • Added tarchive create/update support, so no need to create them via CLI for simple archives. There are known issues with the library used, so it is experimental, but create works well enough for AntTP to use them.
  • Added a postman collection for all the REST endpoints. This helps folks experiment with the API and also allows newman to run a basic test suite. I’ve added this to the pipeline too, to avoid breaking changes slipping through the net.
  • Added mockall and created mocks for the caching clients (which have now been decomposed into many, instead of one, which makes testing easier). It was fiddly to get right, but getting good results now.
  • Added PNR zone update support, so records can be updated with new sub-names, addresses, TTLs, etc. Also added append (as REST PATCH) to simplify adding/replacing existing records.
  • Improved HTML directory listings. Nothing too fancy, but instead of very basic HTML, it now has some CSS styling and some different icons, etc. It looks much fresher, imo.
  • Added key/value support REST endpoints (create and retrieve only, but update/delete coming soon). This builds on PNR, but works with public data to make it simple to use. You just provide a bucket name, an object name and some base64 encoded content and it will store it at that location. Doing a get with bucket and object names then returns the data. No XOR or long private key addresses needed! This should be very handy for app builders!
  • New documentation, with cleaner styling and broken down into chapters. I’ve also added some sequence diagrams and information about the data type flows.

So, progress has been rapid and I hope you enjoy the new features. More coming down the pipe too! :slight_smile:

Github: Release v0.24.0 · traktion/AntTP · GitHub
Dockerhub: traktion/anttp - Docker Image
Cargo: cargo install anttp

16 Likes

@Southside reported that he couldn’t cargo build due to a proto-compiler dependency. proto-compiler is needed to build the template structs from the proto files.

I’ll see about adding a compiler flag to ignore if the dependency isn’t present or try to find a way to get cargo to do it using a native dependency. EDIT: Created an issue: Integrate proto compiler into native Rust build · Issue #92 · traktion/AntTP · GitHub

For now, if you are debian/ubuntu/mint based, installing this first should fix it:

sudo apt-get install protobuf-compiler
3 Likes