This week we round off our look at how we can validate the sections and even the network itself using BLS signing keys to create a link all the way back to Genesis, with the current membership tied to the section prefix via the PrefixMap, and how this is regularly updated by AntiEntropy (AE).
General progress
@Joshuef is seeing some good progress with the memory issues, some of which may have been due to locks coming from traceroute. Together with some DBC test improvements from @bochaco, CI is starting to look healthier again.
@Bochaco is also looking at h3 and WASM and how they might be integrated into the client. There’s a good video on h3 (HTTP/3) here - nice find @19eddiejohn75.
@Roland has written a PR to move SectionChain into the PrefixMap (see below). It will need some testing first.
And @oetyng has two large changes in progress: data storage extraction and simplification of the send message flow.
Data storage extraction
Moving towards a simpler node struct, we’ve previously extracted the comms module. Now we’re on the data module. This was initially in the drive to remove locks within the node struct, but we’re looking at more benefits. The decoupling allows running these components in isolation, making it easier and faster to test them. Also, we can set up smaller versions of a network, combining some modules and not others. For the node struct, we’re moving most I/O out of it, which is one step closer to a potential “pure function” API of the node. This aims to make it easier to reason about the code and the effects of changes, reducing risk of bugs.
Send msg simplification
This refactor removes more than 400 lines of code and reduces the number of places where we call various send functions with different parameters. Those parameters turned out to not be needed, and the different send functions were not needed either, when reworking things a bit. Also, this places signing and instantiation of the WireMsg struct at one place.
How it all fits together
More on BLS
We use distributed key generation (DKG) as an efficient way of reaching agreement between elders. Once we have aggregated five out of seven shares we create a new section key, which is a public key, and also a virtual private key which does not actually exist, because this is an ownerless system. We don’t want anyone to be in charge when we’re reaching agreement.
But each individual elder - or indeed any node - can also use its own BLS keypairs to create any number of key shares, and these can be used in a dealer-based mechanism. In this system, there is an owner who controls the secret key, and that owner is free to create and distribute shares in an n-of-m system. For example, the owner of a website might distribute key-shares to three trusted employees, and require at least two to sign off any changes to the site.
We wouldn’t use a dealer-based mechanism for decentralised consensus as we specifically don’t want there to be an owner of the secret key, but we can use it to implement data sharing.
More on the PrefixMap, SectionChain and SAP
The PrefixMap is simply a structure that maps all the section prefixes (00, 01…) to their current section authority providers (SAPs).
The SAP is a list of elders in the section, with their IDs, IPs and ports, plus the current BLS section key. So it provides all the information a node or client needs to join the network.
PrefixMap
Prefix SAP
00 → {[A(IP, port), B(IP, port), ….F(IP, port)], Sec_key}
01 → {[G(IP, port), H(IP, port), ….M(IP, port)], Sec_key}
….
Every node and client has a copy of the PrefixMap – they’re sent one as soon as they connect, or they may obtain one from friends, or wherever. But not everyone’s PrefixMap is up to date, so the client or node making contact with a section must first update its PrefixMap with the current SAP through AntiEntropy (AE). Only when it has an up-to-date SAP can it start to do business with the section.
But, of course, it also has to know the PrefixMap is genuine. For this, it needs to check the SAP is signed by a valid section key (the current key minus 1), and also that that section key is valid, because it has been signed by the one before, etc. All the way back down to the Genesis key.
So the other part of the PrefixMap is the SectionChain. This is a binary tree structure that links all BLS section keys all the way back to Genesis. Currently this is a separate element and we sync between the key in the SAP and the chain, but we plan to bring them together – at which point we’ll have to think of some better names!
At that point the PrefixMap/SectionChain, or whatever we eventually call it, will be a DAG with the Genesis key at the root and the chain of SectionKeys and the current key and membership at the leaves.
The other thing to mention is that if a client or node reconnects to the network, it doesn’t need to update the whole SectionChain again, just the part that’s new. So the Section it’s connecting to sends that new bit of the chain – called variously a ProofChain or chain part – to the reconnecting node so it can simply add it to its existing SectionChain.
With this design, even if someone should succeed in stealing the Genesis key and impersonating the network, this would matter less and less as the network grows larger and larger and the DAG has more and more branches, because we will have to trace the keys all the way back to Genesis again less and less frequently.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
