Summary
Here are some of the main things to highlight since the last dev update:
- Our adoption of proptesting has been bearing fruit with some important issues identified this week.
- Significant performance improvements to the network startup process have been merged here, with further proposed improvements in progress here.
- Our CRDT consultant has begun working with us again this week and dived straight into enhancing Deterministic Secure Broadcast (DSB) with support for dynamic membership.
- Kudos to forum member @treslumen for their contribution to
sn_apihere.
Safe Client, Nodes and qp2p
Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan
We’ve been shoring up our Sequence data this week, starting with various proptests to ensure the CRDT data converges as expected under various different circumstances. We’ve unearthed a possible bug in how permission policies were being applied and have been working on a fix, plus more specific tests for that particular area.
Alongside, we’ve started investigating how we can use the Sequence Data’s permission system for Map data, with the aim of making it CRDT ready. Initial investigations indicate it’s looking like we can apply this wholesale, so right now we’re setting up the basics for that and, assuming no blockers, we’ll start firing in with tests afterwards.
On sn_node this week, we’ve been zeroing-in on an inter-section message passing bug that intermittently prevents clients from uploading data to the network, highlighted by failing data tests in our internal testnet. All of the network’s messages maintain a list of entities/authorities that it has traversed to reach its destination, which wasn’t being updated correctly for its reply to come back. A refactor and a fix is underway for this and, once squashed, it should put us back on track to continue pushing the testnet forward.
We’ve also happily accepted a PR from forum member @treslumen, removing some obsolete code after the recent sn_client updates. Thanks for that @treslumen, very much appreciated 
Transfers
Continuing on from last week, there has been further work on improving anonymity and consistency.
We’ve also returned to look deeper at Digital Bearer Certificates (DBCs) and see how we can use them now that we’ve made a lot of progress with CRDTs and AT2.
A DBC is a mechanism whereby a token or certificate can be held and then used. What DBCs can give us, if we manage to solve a couple of things, is full anonymity and offline transactions.
CRDTs
This week our CRDT consultant has begun work to enhance Deterministic Secure Broadcast (DSB) with support for dynamic membership. Dynamic membership is the process by which new peers may securely join or leave a peer group (section) by voting/agreement of existing members.
The leave case is most interesting because it entails detecting when a peer is faulty and then other members vote to exclude the faulty peer. A peer may be considered faulty for many reasons, including becoming non-responsive, using the wrong protocol, sending bad data, or sending invalid signatures.
As a reminder, DSB is a Byzantine fault tolerant (BFT) protocol that is presently used for securing AT2 transfers. It is also useful for securing CRDT algorithms.
It is expected that DSB with dynamic membership will form a solid foundation for future routing improvements as sections must regularly agree on adding and removing elders and other nodes.
Routing
To ensure routing level tests are using the same network setup as the planned testnet configuration, we decided to remove NetworkParams and use const value for testnet. This enforces the use of the same constant values for a section’s size and a section’s elder_size, across the routing and node level tests.
This week we also tweaked the network startup process to improve the startup performance. We replaced the DKG unknown message bounce-back with a simple backlog during the startup, reducing the time taken for a first section to fully form with all its elders from minutes to seconds. This also adds a proptest for DKG to further ensure the code quality and network reliability.
Further improvements to the network startup phase have also been proposed in this PR, which is currently working through testing and peer review. During the startup phase, every infant that joins is instantly relocated back to the same section, but with increased age. This changes the process slightly so that the new age is pseudo-randomly generated from a given age range, therefore preventing the peers from having the same age, which would cause them to be relocated at the same time. It also updates the relocation calculation so only the oldest nodes are relocated.
As mentioned in the Nodes section above, an issue was found on an internal testnet where the payload of section-to-section messages from nodes to routing were not always identical across elders and clients. It was initially thought that this was a bug in the routing layer, but after some investigation, we traced it back to nodes and this is now in their capable hands.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
