:IF: Ark - Backups (mostly) done right

[RolandR]

To the forum I humbly present my IF submission:

Ark - Backups & Archives That Stand the Test of Time

Impossible Futures Project Page | Github Repo

WTF is Ark?

Ark is a system for backups and long-term data archival that is designed to be secure, lasting, self-sovereign, highly flexible but also practical at the same time. A lot to ask at once .

It’s carefully structured in a way to fully take advantage of what Autonomi offers while hopefully avoiding most of the drawbacks. I am also trying to make it appealing for non-technical users and people / businesses / organisations from outside the decentralized / web3 ecosystem without alienating the more “hardcore” users. That’s even more to ask from a design . But I’m hopeful.

How it works, very (very) roughly

There are two big design decisions that truly set Ark apart:

1. Key Hierarchy & Separation of powers
   Keys (and responsibilities) are split into distinct roles (Data Key, Helm Key, Worker Key). Key recovery and rotation - which is often overlooked - is easy and straightforward. The Ark Seed is the ultimate master key.

2. Engine/Bridge Architecture
   A Bridge is a lightweight go-between that connects any data to the Ark Engine. If a bridge exists for a data source, we can back it up. The Ark Engine does most of the heavy lifting and can be run somewhere or even by someone else (it never sees plaintext data).

I’ve tried to explain this in more details on the project page and the Github (for a more technical take). Naturally, Ark also takes full advantage of Autonomi’s global deduplication and pay-once-store-forever - but that’s a given.

About Me

I’ve been around the block for a bit and up & down the tech engineering ladder - as well as sideways.
In the end I’m most driven by imagining how things can be done better and then actually making it happen.
I spend a lot of time thinking about how things interact - technically, but also the non-technical aspects of it. I like to say that I think in systems - which, frankly, sounds a bit pretentious (because it is). Why are (or aren’t) people doing things in a certain way? How does the problem look from their perspective? What is limiting them? What are their needs? How are the incentives currently aligned. Besides the technical stuff, there are also financial, legal and practical aspects that need consideration when designing systems (even autonomous ones).
In the end, the systems we build don’t exist in a vacuum and are a means to an end (something us devs tend to forget occasionally).

Project History

Ark is my current attempt of tackling a problem I’ve wanted to deal with for a long time. I am not happy with the existing backup solutions that are out there. None of them come close to how I think things should be. I’ve made some smaller attempts over the years to build something better - but something was always missing to truly get it sorted. Autonomi might just be the missing piece of the Tetris!

Onward

I’ll keep working on this during the whole voting period. And I’ll keep pushing releases and updates and will let everyone who’s interested in Ark’s development arc know how it’s going.

If you like what I am doing I’d love to hear from you. I’d truly love to talk more about the nitty-gritty of Ark - a lot of thought went into it!

Impossible Futures Project Page | Github Repo

[RolandR]

I’ll post any update here.

Last Update:

08/May/2025

It’s Alive!

v0.0.1 is out:

The very first public code for Ark is out - v0.0.1!

This Proof-of-Concept comes with a CLI tool that lets you create an Ark from scratch and experiment with key rotation. Not much more in there yet from a user’s point of view, but it demonstrates the core concept with its key structure and management.

Impossible Futures Project Page | Github Repo

[RolandR]

Last Update:

14/May/2025

Not Slowing Down!

v0.0.2 is out:

The next release of Ark is out - v0.0.2!

It’s still early days, but the foundation is getting stronger and the pieces are falling into place. There’s been a lot of progress under the hood in just a week - the cryptographic components are improving and things are getting more flexible.

A couple of new user-facing things to play with:

• Ark Details: The CLI now lets you see detailed information about an Ark using any of its authorized keys - Helm Key, Data Key, Worker Key - or the Ark Seed
• Vault Creation: You can now create vaults directly from the CLI. This includes setting which bridge is authorized and what type of object the vault will handle: filesystem (POSIX/Windows), email (IMAP/Gmail), or object storage (S3).

Overall development is progressing really well! Soon, work will start on the Ark Engine. Once a basic version is out, Arks premise will start to become much clearer and more self-evident.

Impossible Futures Project Page | Github Repo

[RolandR]

Developer’s Log

Stardate -298031.96 14/May/2025

The past few weeks have been exceptionally busy, but also highly productive. Ark has moved fast: from an idea, to a concept, to a proper IF project, and now to a proof-of-concept (emphasis on proof).
Time for a bit of reflection:

What went right

• Product Design: I put a lot of thought into Ark’s design, and it paid off. It addresses every challenge I set out to tackle, and stands out from anything else I’ve seen in this sector. And it really takes advantage of the strengths of the Autonomi Network.
• Rust Code / Structure / API: Ark’s code takes full advantage of Rust’s type system. It tames complexity and eliminates entire classes of bugs. This is one of my cleanest codebases so far, and if I can maintain this quality, it will pay off in the long run.
• Brand Design: The brand design landed well, giving Ark a strong visual identity that stands out. It’s a solid starting point for the future.

What went wrong

I feel like I utterly failed at getting the point across why Ark matters as much as I think it does.
I’m significantly better at building the thing than I am at writing about it - and that really shows! All that pretty LLM generated text that no one really reads …

Fortunately, there are folks out there who do get it, perhaps even despite my efforts. Like this Discord quote:

I have thrown a lot at ARK because I NEED THAT. And imo it’s the only project that will deliver a product that I can wave at enterprise IT people as an example of what autonomi is and can do for them.

What’s Next - The Beginning of a Roadmap

Right now, Ark isn’t very tangible for a user just looking at it from the outside. That’s going to change soon. The next big push is building out the core Ark Engine . Once a basic version of the Engine exists, work will start on the first Bridge . As soon as we have both, a working Engine and a Bridge, I’ll begin developing the communication protocol between them.

I’m especially excited about this Engine<->Bridge protocol. If it works out as planned, we will end up with a solution that is as easy (or even easier) to use as a regular, fully managed cloud service but WITHOUT ever giving up control or privacy! Stay tuned!

All of this is just weeks away, not months. If Ark moves to the next IF phase, a fully working product should be ready within the IF timeline. I’ll also release a much more detailed roadmap soon.

I want YOU!

I’m genuinely excited about where Ark is heading, but I can’t do it alone. A sincere Thank You to everyone who’s supported Ark so far.

If you’re still going to vote, please consider Ark - even if it’s not something you personally need right now. Backup and archiving are a massive industry and - frankly - a huge pain to deal with.

There are many people out there (especially business users) who are willing and able to pay good money for a better solution. And they bring a lot of data with them - exactly what the Autonomi Network needs right now!

Thank you.