Announcing - MutAnt: Mutable Key/Value storage (prev: Anthill)

MutAnt: Easier Mutable Private Storage on Autonomi!

Hey Autonomi community!

I’m excited to share a project I’ve been working on called MutAnt. If you’ve ever wanted a simpler way to store and manage private, changeable data using Autonomi’s storage capabilities, MutAnt might be for you!

What is MutAnt?

Think of it as a friendly layer on top of Autonomi’s Scratchpad features. Instead of dealing directly with the lower-level details, MutAnt gives you a straightforward key-value store (like put, get, remove) using easy-to-remember string keys. It’s built to be asynchronous, so it plays nicely with modern Rust development using async/await.

Why did I build this?

My goal was to make interacting with Autonomi’s mutable storage more convenient, both for command-line use and for integrating into other Rust applications.

Key Features:

  • Simple Key-Value API: Store, fetch, and delete data using mutant put mykey "my value", mutant get mykey, etc.
  • Human-Readable Keys: No need to juggle complex identifiers; just use strings.
  • Async-First: Designed for non-blocking operations.
  • Progress Updates: Get feedback during potentially long store and fetch operations (especially useful in the CLI).
  • Handy CLI Tool: Comes with the mutant command for quick interactions.

Want to try it out?

Heads Up! MutAnt is still under active development. It’s not ready for mainnet or production use yet. Expect things to change, and use it carefully, especially on testnets for now.

Installation (CLI):

The easiest way is via crates.io:

cargo install mutant

Or you can build it from the source:

git clone https://github.com/Champii/MutAnt.git
cd MutAnt
cargo install --path .

Quick CLI Examples:

# Store some data
echo "My secret message" | mutant put my-secret-key

# Retrieve it
mutant get my-secret-key
# Output: My secret message

# List your keys
mutant ls

# Delete a key
mutant rm my-secret-key

(Use mutant --help for all commands and options, like using a local testnet with -l)

Using it in your Rust Code:

You can also add mutant_lib to your project:

# Cargo.toml
[dependencies]
mutant_lib = "0.1.0" # Check crates.io for the latest version
tokio = { version = "1", features = ["full"] }
// Basic example
use mutant_lib::mutant::MutAnt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Needs your private key hex string
    let private_key_hex = "0xYOUR_PRIVATE_KEY_HERE";

    // Connect (use init_local for local dev)
    let (mutant, _) = MutAnt::init(private_key_hex).await?;

    mutant.store("my_app_data", b"some bytes").await?;
    let data = mutant.fetch("my_app_data").await?;
    println!("Fetched: {:?}", data);

    mutant.remove("my_app_data").await?;

    Ok(())
}

Development & Testing:

If you want to hack on MutAnt or run the tests, you’ll need a local Autonomi testnet. Check out the scripts/manage_local_testnet.sh script in the repo – it helps set up and manage one for you. Details are in the README!

Find out More:

I’d love to hear what you think! Feel free to try it out, check out the code, and open issues or suggestions on GitHub.

Cheers!

24 Likes

Another amazing app on its way to prime time. I can see this being useful indeed. This is top layer, correct? Where data can be added and removed? Would the data removed no longer be permanent? Network app dev is not my area of expertise. I’m a game dev. Thanks

11 Likes

Hey thanks !

Yes this allows you to completely manage a defined space of storage. Once deleted, a content cannot be retrieved anymore, and that space become free for more data without additionnal supplement.

8 Likes

I see. Interesting. Yes, I can imagine this be popular and useful, in many instances. Good work.

8 Likes

I think it has been mentioned before, but having a 4MB space that can be reused as many times as desired by only paying the price of a chunk couldn’t it be considered a potential attack vector on the network?

8 Likes

Yes I made that exact same argument in my first Anthill post.
There has to be a cost to mutate the network or we will see abuse.

Well, either I’m wrong and the network creators know what they are doing, or maybe such things will be fixed in future updates who knows :slight_smile:

5 Likes

At least it can lead people buying storage at the lowest price point with any junk data and the (re)filling whenever.

2 Likes

In fact, for storing private files smaller than 4MB, using Scratchpad instead of Chunks has multiple benefits. It costs a third as much since the file isn’t split, as happens with chunks. And you can delete and modify it as many times as you want.

With the right management program, chunks and self-encryption lose much of their usefulness.

3 Likes

But it has no history and just re uses the same space - so the ongoing cost for updates is just traffic - traffic comes with immutable data too

Aaaaanyway

This is a false statement - there’s discrimination between data types in the node code for charging different amounts for different data types - if it isn’t used right now then it’s due to the network having bigger issues than the precise (neglectable) amount of ant being charged for one data type or the other :man_shrugging:

2 Likes

Even now you can use your own storage method as long as no immutable record is not larger than 4MB. The self encryption is just part of the standard client. Using the api or your own client you can use whichever method.

Node encrypt the records before storing as well and decrypt when serving up the record, so at rest the data is encrypted still.

3 Likes

It’s the doubt I have, but looking at the code, I’m not clear on this difference.

Is there currently any difference in price between a Chunk and a Scratchpad?

Yes, but using Chunks, you can neither delete nor mutate them, which is possible with Scratchpad, greatly increasing its utility.

6 Likes

Sorry on mobile and only single handed atm - difference is not part of the Scratchpad code but the node - last time I checked the calculate cost for record function did expect the data type to be passed in iirc

2 Likes

Hey guys !

I’m really excited to share MutAnt v0.2.0 with you all! I’ve been working hard on this update, packing it with features to make storing and retrieving your mutable key-value data on the network smoother and safer.

What’s MutAnt again?
For those who haven’t seen it, MutAnt is a project I built to provide a user-friendly layer on top of Autonomi’s scratchpad storage. Think of it as your personal, private, and mutable key-value store directly on the network, accessed via a simple command-line tool (mutant) or a Rust library (mutant-lib). Store text snippets, config files, small assets – whatever bytes you need kept private and accessible!

Why I’m Excited About v0.2.0 (What’s New?)

This release tackles some of the biggest hurdles I found in previous versions, focusing heavily on reliability and user experience. The feature I’m most proud of? Resumable Uploads!

  • Never Fear Network Drops Again! (mutant put): It’s happened to me, and probably you too – you start uploading a file, the connection hiccups, and… frustration. With v0.2.0, MutAnt now cleverly saves the progress of your uploads. If it gets interrupted, simply run the same put command again, and it will pick up right where it left off, verifying data checksums to ensure nothing got corrupted. This brings massive peace of mind, especially for larger values.
  • See What’s Happening (Clearer Progress): I’ve revamped the progress indicators for commands like put and sync. Multiple detailed bars now show you exactly what’s happening under the hood – reserving space, transferring data, confirming writes, and synchronizing state. No more guessing!
  • Rock-Solid Syncing (mutant sync): Keeping your local cache aligned with the network state is crucial. The sync command is now much more robust, handling edge cases like missing remote indexes gracefully. I’ve also added a --push-force flag for situations where you need your local view to be the definitive one.
  • Storage Management Tools (mutant import, mutant purge): Got an old scratchpad lying around you want to add to your MutAnt pool? Use mutant import. Need to clean up storage attempts that got stuck? mutant purge helps tidy up pads that were reserved but failed verification.
  • Snappier Experience (Lazy Initialization): Commands that only interact with your local cache (like mutant ls or mutant stats) now start much faster, as MutAnt waits until it actually needs the network to establish a connection.
  • Squashing Bugs & Improving Stability: I’ve dug deep to fix various hangs, potential data consistency issues during concurrent operations, and improved overall error handling. Verification loops now have better timeouts and retry logic for greater resilience against temporary network issues.
  • Smarter Caching: Your local index cache is now kept separate depending on whether you’re interacting with Mainnet or a local Devnet, preventing accidental mix-ups.

Using MutAnt v0.2.0 on Mainnet - A Word of Caution

This v0.2.0 release represents a major step forward in stability and safety compared to previous versions. The resumable uploads and data integrity checks significantly reduce risks during data transfer.

However, I want to be transparent! While I’ve made great strides, the underlying logic for reserving and allocating storage space (the scratchpads) is still an area I’m actively developing and refining.

Therefore, my recommendation for mainnet usage is this:

  • Storing smaller key-value pairs, configurations, or important snippets should be considerably safer now. Go for it!
  • I’d advise exercising caution with very large files or performing a high volume of rapid put/rm operations for the time being. The allocation logic might still have edge cases under heavy load that I haven’t caught yet.
  • Stick to data where the improved reliability of the transfer process (thanks to resumable uploads) provides significant value, but where potential (though less likely now) allocation quirks wouldn’t be catastrophic.

Ready to Try It?

I’d love for you to give MutAnt v0.2.0 a spin! You can install or update easily via crates.io:

cargo install mutant --force

Or, if you build from source:

git fetch --tags
git checkout v0.2.0 # Or pull master if you prefer the latest dev changes
cargo install --path . --force

Find Out More:

I’m really excited about this release and the foundation it lays for future improvements. I hope you find MutAnt increasingly useful! Please share your experiences, report any issues you encounter on GitHub, and let me know what features you’d like to see next.

Happy Storing!

16 Likes

Hey guys !

I’m happy to share that I’ve just released version 0.3.0 of MutAnt (mutant-lib and mutant-cli)!

You can install it with cargo install mutant

This release focuses on making MutAnt more robust and user-friendly. Here are some of the main improvements:

  • Resumable Uploads: The resume feature is more robust now. It also tells you if a key is incomplete in ls and stats.

  • Better Reliability: I’ve added retries for some network errors and improved how data consistency is checked during uploads.

  • CLI Improvements: There’s a new mutant reserve command, better progress bars, and put is smarter about handling existing keys.

  • Lots of Fixes & Refactoring: I’ve cleaned up the code, fixed various bugs (including some related to network operations and index handling), and added more tests and documentation.

You can grab the latest versions from crates.io:

For the full details, check out the CHANGELOG.md.

Also, a reminder that while it works perfectly on the testnet, the mainnet - at the time of writing - is struggling with scratchpad uploads > 3MB, so for maximum reliability I would advise to not go above that limit when uploading on mainnet (for now)

I’m open for bug report and feature requests :slight_smile:

Cheers

12 Likes

Hey guys !

Ok so I’ve now release the v0.4.0 of MutAnt that introduces Public Mutable uploads and downloads.

It means that you can now upload data to the network that is public and mutable only by you, and the public access key stays the same.

As a demo, I will be running a very simple loop that will upload a public mutable message to the network for you to try.

You can update your mutant to the latest version with:

$> cargo install mutant

Then try to get the public data:

$> mutant get -p 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980
#output: Hello Autonomi ! Sat, 19 Apr 2025 14:38:44 +0000% 

You dont neet to setup any wallet or private key to use the get -p command.

Details

On my side, i’ve done this:

$> mutant put -p time placeholder
#output: 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980

I can see my public key in my local keys and act on them like i would do with my private keys.

$> mutant ls
#output:
#test_1
#private_data
#time (public @ 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980)

Then I run this rust code:

use chrono::Utc;
use log::info;
use mutant_lib::MutAnt;

const PRIVATE_KEY_HEX: &str = "0x__REDACTED__" 

#[tokio::main]
async fn main() {
    env_logger::init();

    let mutant = MutAnt::init(PRIVATE_KEY_HEX.to_string()).await.unwrap();

    loop {
        let output = format!("Hello Autonomi ! {}", Utc::now().to_rfc2822());

        mutant
            .update_public("time", output.as_bytes())
            .await;

        info!("Successfully updated the time");
    }
}

You will find more information on the MutAnt GitHub repo

Show me what time you got ! :slight_smile:

8 Likes

It JustWorks!!! :tada:

willie@gagarin:~/projects/maidsafe/AlphaNet$ mutant get -p 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980
Hello Autonomi ! Sat, 19 Apr 2025 15:06:45 +0000
9 Likes
$ mutant get -p 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980
Hello Autonomi ! Sat, 19 Apr 2025 15:31:15 +0000
5 Likes

Very nice !

Now i’m working on reducing the number of public scratchpads for small data < 4MB because right now even if you upload 1B of public data it reserve 2 scratchpads so 8MB (not ideal)

I also want to add a feature to the CLI to process folders recursively and have a kind of a FS structure with the keys. That would open the way to a FUSE driver to mount mutant directly on your fs (that’s a far away dream, just to tease you a bit :p)

4 Likes

Ok so I did a small update (v0.4.1) with a Quality Of Life feature that keeps an history of the public keys you fetch in ~/.config/mutant/fetch_history.cbor (to be changed later to ~/.local/share/mutant/.
You can access it in the mutant ls -l command

$> mutant ls -l
SIZE TYPE    MODIFIED     KEY/NAME
3 B  Public  Apr 19 02:05 a @ 90ec4fa58c95068c462f6158986006288464abd0eee7840171ad562a0ce615e178ed28172e8273e8c5b385c69bb37e43
1 B  Private Apr 19 13:57 b
48 B Public  Apr 19 16:59 time @ 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980

--- Fetch History ---
SIZE TYPE     FETCHED      ADDRESS
48 B Fetched  Apr 19 16:51 9429076971abe17b485fd30dd3065d27fc36362ba164529e530722bdd693f6cb8904fc177bf657d29774eb42403ac980
3 Likes