Summary
Here are some of the main things to highlight since the last dev update:
- A new set of Infrastructure messages which
sn_routingcan use to better handle and return errors during section splits and churn has been added. - Work to handle invalid SignatureShares by bad actors in transfers has begun, with penalties for those bad actors now being drawn up and agreed upon.
- Messaging flow work is nearing its completion, meaning significantly less code and complexity to analyze incoming messages in
sn_node, and clearing the way for the integration of rewards. - New versions of the CLI (v0.19.0) and
authd(v0.1.1) have been released, making them compatible with the latestsn_node(v0.26.16), as well as including several other improvements. - The very first example app is now available in
sn_apicrate, demonstrating how to upload a file to the network and then fetch it using its XOR-URL. - Weāve been making steps towards integrating
sn_fsandBRBso as to create a P2P byzantine-fault-tolerant distributed filesystem prototype. sn_routingsection chain modifications to resolve forks are mostly in place now, with our testing indicating quite a bit of improvement to the stability of the network, especially around splits.-
Yet more fantastic community contributions to the code base
Testnet status - on hold
As discussed last week, we decided to switch our full efforts to include rewards in the next testnet. Rewards itself has been blocked by related work to adjust the message flows after we made the switch to only do message accumulation in sn_routing. We believe we are in the final stages of the message flow work now, with what we believe are some final kinks being ironed out (more details below). Once done, this clears the way for the full integration of rewards to proceed.
As with all groundbreaking progress, we expect that there will be issues found during the integration and testing stages, so we donāt want to put a timescale on that and put additional pressure on the team. As always, we will keep you up to date with precise progress in these weekly updates.
Thank you for your patience!
Safe Client, Nodes and qp2p
Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan
Lazy messaging
To better handle issues at section split, and during churn, weāve been adding a new set of Infrastructure messages which sn_routing can use to return errors such as DKGINprocess when we attempt to message elders during such times. In order to do this, clients/nodes will supply the current PublicKey for a section, as far as they know, and we can take action if that is, for example, out of date. These changes have been added into sn_messaging, sn_client, and sn_routing, and weāre looking at some small refactors to simplify things with nodes.
On the back of this, weāve been further solidifying client handling of such infrastructure changes, so a change in the pipeline will allow clients to re-bootstrap any amount of times as is necessary. Alongside this, we are also squashing bugs that are arising with the message accumulation refactor done over the previous weeks.
Penalising bad actors
Handling invalid SignatureShares by bad actors in transfers is also in the works. Up until now we have been approving a transfer whenever valid threshold+1 shares were aggregated successfully, ignoring any faulty shares that arenāt necessary for that aggregation. Weāll now be introducing penalising mechanisms in the network to punish nodes accordingly after verifying. We are beginning our refactor from sn_transfers, as that is where accumulations happen for payment transfers, then working our way up through the layers.
Messaging flow
When moving over to accumulation in sn_routing only, we needed to remove unique info in messages, so that Elders would sign the same message.
With this the source and destination pattern had to be changed and while at that we made it simpler, by slimming the messaging interface between sn_node and sn_routing, and using the same primitives over the boundary.
As client bootstrapping is now taking place in the sn_routing layer, these location primitives had to be extended with a client variant. A registry of connected clients was implemented in sn_routing together with providing public key and signature over socket address at the client bootstrapping. There is now significantly less code and complexity to analyze incoming messages in sn_node. So far a total of about 1000 lines have been removed.
API and CLI
New versions of the CLI (v0.19.0) and authd (v0.1.1) have been released this week, which includes all the fixes, improvements and new features implemented since the previous release. As usual, they can be upgraded with $ safe update and $ safe auth update respectively. These applications are compatible with the latest sn_node version v0.26.16, so for those upgrading please also make sure you upgrade your sn_node binary as well ($ safe node install or $ safe node update).
@scorch has spotted that the very latest version of clippy was complaining on Windows for the CLI and authd codebase, and submitted a fix for it. This led us to improve the clippy checks in our CI scripts to run not only on Linux but also on Windows, which should prevent this from slipping through again.
With the addition (by @mav) of a new API in our CRDT Sequence data type to fetch an entry specifying a single index rather than a range, @mav now changed our sn_api to make use of this new API for retrieving a specific version of a Sequence.
A couple of enhancements have been also made to this new CLI (v0.19.0) which allows the user to set up bootstrapping information for a network using an IP (v4 or v6) and port number, for more details of this new command please refer to the corresponding section in the CLI User Guide.
For those curious to see how the Rust API is evolving and what it will look like to build an application with it, a very first example app is now available in sn_api crate which demonstrates how to upload a file to the network and then fetch it using its XOR-URL. Hopefully this will inspire others to create other simple example applications for our Rust Safe API, find out improvements opportunities as we start using them, as well as be a good resource for new devs willing to write Safe applications.
CRDT
David Rusu, author of rust-crdt as well as our BRB implementation, has been reviewing the LSeq code after @mav reported issues with it when many inserts are performed. He replaced the ID implementation that was based on the original LSeq paper with a rational number from the num crate. This makes the LSeq code much simpler and also results in a better (more even) distribution, resolving the problem and allowing the insertion of an arbitrary number of items. LSeq has also been renamed to List. Pull Request.
BRB - Byzantine Reliable Broadcast
Remember sn_fs, our prototype of a filesystem using the tree CRDT? Well this week weāve been making steps towards integrating sn_fs and BRB so as to create a P2P byzantine-fault-tolerant distributed filesystem prototype.
First we forked brb_node_qp2p and created brb_node_tree, which demonstrates sending crdt_tree operations via brb. Then we modified the sn_fs crate and turned it into a library. Most recently we created a brb_node_snfs crate in which we bring everything together. As of today, we have been able to bring up two (or more) nodes and issue directory operations such as mkdir, rmdir. The operations are mirrored in the mounted filesystem of each peer. File operations are not yet working, but should be soon.
It should be stressed that this is bare-bones proof-of-concept work. There are many details to be worked out before it could be put into real usage.
Routing
The section chain modifications in order to resolve forks, as discussed in last weekās update, are mostly in place now. We are currently running internal tests using the stress-test tool to identify any remaining issues. It seems the stability of the network has improved quite a bit, especially around splits. Some issues still remain however and we are busy debugging them, but overall we are optimistic about this approach.
We also merged a PR to fix a message explosion, this happened when an elder went offline but the remaining elders would still attempt to send them the Offline vote, which would of course fail to be delivered and so would then trigger further Offline votes.
We merged another PR to respond with error to a client request if there is a DKG ongoing. An ongoing DKG (Distributed Key Generation) means that the section is about to transition to a new set of elders and so things might not be entirely stable or consistent at this time, therefore itās better to let the client know so they back off and try again a short time later.
[UPDATE 22nd Feb] - Safe Browser
Something we originally forgot to add to this update 
@bzee has created a PR to get sn_nodejs up to speed with the latest API changes, which is needed for the Safe Browser to be updated to be compatible with the rest of the Safe landscape again. He has also been taking a look into the limitations of the current lib, and seeing opportunities to improve things!.
Thanks @bzee! 
and apologies for originally missing your contributions in the OP.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French ; 
Bulgarian
As an open source project, weāre always looking for feedback, comments and community contributions - so donāt be shy, join in and letās create the Safe Network together!
and a very nice update again, thank you everyone working on this. Things are still flying and Iām happy that the next testnet will include rewards. Even as a first cut that is a very exciting prospect 
). Thank you to 
) and maybe more. That will take us to Beta I hope. Letās see though.
this next testnet sounds like it will be the real deal. Take your time and get it right. Super update 
keep it up 
