Internals of the Safe network

Hi everyone,

I have been lurking around this exciting project for a while, and I’m now trying to understand the internals of the Safe network.

I have read the Primer (latest, Nov 2021) and most weekly updates, but I need help understanding some things. I would appreciate any help.

1. Hops and proxy

As far as I understand, the client never directly interacts with Adults and Elders. Instead, it connects to a random node, and all communication with the network happens via this node. This node works as a proxy.

Because of this proxying, I assume the client never knows the IP address of the Elders or the Adults it interacts with, and vice-versa. Is this right?

Is there a specific term for this type of node mode? “Exit node”, “Proxy node”, “Guard node”?

Are these proxy nodes always Adults or always Elders? Can they be of any type?

When a client requests data (GET), do the Adult nodes send the data to the proxy or directly to the client?

What about PUTs? Does the client send binary data directly to the Elders/Adults? Or is it via a proxy node?

And communication with the Elders? Is it direct or always via proxy? Is it always the same proxy node? Can the client use many proxy nodes at once for performance?

Is it just one hop? Is there a minimum, a max, or a fixed amount of hops? I assume we need at least two hops between the client and the final node to obfuscate both IP and intention.

2. Who receives the PUT data? Elders or Adults?

When the client wants to store data, it asks each Elder for authorization (via proxy, I assume). Once it gets a supermajority, it sends the binary data - along with the authorizations for proof.

To whom the client sends the data? To the Elders or the Adults?

3. Replication

When the client finishes sending binary data, I assume it gets a response that everything went well.

Is this response after the replication process has finished? Or the client gets a return before the replication?

How many copies a chunk has?

4. Chunks

How do the Elders keep track of who stores each chunk? Do they keep a database with the complete list of chunk-xor/adult-xor? Or is inferred, based on the Xor proximity?

5. Anti-Entropy

When the Elders vote to add/remove an Elder, the AE system makes the section stop accepting mutation requests.

I assume this pauses any request for altering the Section structure (i.e., add adults, infants, elders). But does the Section stop sending GETs and accepting PUTs too?

6. Section Merge/Splits thresholds

On average, a section has about 100 Adults and 7 Elders, according to the Primer.

I understand these values are not fixed, as the network is volatile - the node configuration of a section depends on churns, merges, and joins.

But is there a threshold to soften the merge and split operations? Something like “network splits above 120 Adults and merges below 50 Adults”? If so, what are the values?

And what about the Elders? Is seven a fixed number that the Section tries to reach (ex., always the seven oldest adults in the Section), or does it uses some threshold too?

7. Infants

I assume an Infant is promoted to an Adult when the farming_rate hits some threshold. Is this right?

If so, it means the network as a whole, not a specific Section, needs a new Adult. Do all the sections start promotions simultaneously due to the threshold hit? Or somehow, the network chooses a few Sections to promote Infants?

Which Infant is picked for promotion? Is it the oldest? How does the Section know who is the oldest? The infants have the age of zero until they get promoted, so the sections cannot use their ages. Is there a queue all Elders keep with the time section registered the Infant, so they know the oldest?

When the Section promotes an infant, it sets the Infant’s age from zero to five and sends the Infant to another section. What if this new Section doesn’t need a new adult?

8. Data types
According to Primer, there are two data types: Blob and Register. Blobs are immutable (ex., image), while Registers are mutable.

A Register can be either a Map (key/value) or a Multimap (key/values).

I know some complex types are built on top of these two data types: data map (for files), NRS (like DNS), but is there any other primary type?

What about private/public data? Is there a flag on the data that specifies the visibility? A different type? Or, to make the data private, do we have to encrypt it before sending it to the network?

What about authorizing multiple people to write to the same Register? Is it possible?

9. Pay to update a Register
We have to pay to create a new Register. But what about updating its value? Is for free or paid?

10. Old terms
Some terms are on the forum but not on the Primer. Do they still exist?

These:

  • Close group;
  • Data chains;
  • Replica/sacrificial/backup chunks;
  • Appendable data;
  • Gossip protocol;
8 Likes

I really cant snswer much of this, but heres a recent discussion where storage was discussed briefly.

3 Likes

Afaik, possibly im wrong, everything is encrypted, public data would just have the map of chunks / decryption keys shared. Pvt data wouldnt.

5 Likes

This is still current AFAIK

When someone makes a GET request, that kick starts a number of processes. First, the request is relayed to the elders in which section the chunk is stored. The elders then calculate which four adults store the chunk - the four XOR-closest to the chunk’s address - and each of those adults pass their copy of the chunk back to the elders, who then return them to the requesting client.

4 Likes

10. Old terms
Some terms are on the forum but not on the Primer. Do they still exist?

These:

  • Close group; - still applies as a concept but not term is not used
  • Data chains; - replaced by SectionTree to take account of forks
  • Replica/sacrificial/backup chunks;
  • Appendable data; replaced by registers - appendable data that allows forks
  • Gossip protocol; still there
6 Likes

There is no proxy at the moment but that will likely be put in place, something like onion routing, at a later stage.

6 Likes

Yes
Don’t know - before I think, although replication will be very fast
Three (for now)

4 Likes

Inferred on Xor proximity

4 Likes

Yes

There will be a max for splits but I don’t know what it will be. Merges are no longer part of the design.

Seven is fixed. Sections will always try to have seven elders.

No this is just a mechanism to adjust the reward. Sections add new nodes as they need them.

Random I think, all infants are age zero.

It has to take it - I guess there’s a reasonable amount of flexibility for the number of adults in a section. I don’t think this has been finalised yet.

Paid.

5 Likes

I see. The proxying is on hold for now, but the goal is to have it in the future.

The proxy will always be an Elder:

Client <-> Elder (Proxy) <-> Elders (Section responsible for the data) <-> Adults

The Adults identity is protected from the Client and vice-versa.

5 Likes

My reading is that the anonymity is still on the to-do list, which may mean using proxy nodes but might be some other mechanism.

3 Likes