Summary
Here are some of the main things to highlight since the last dev update:
- The call for volunteer committee members for the BambooGarden Fund has now closed! We have 8 respected and experienced committee members from the community in place.
- With the BambooGarden Fund committee now confirmed, we will move on to setting up communication channels and being able to invite the first fund applications.
- We’ve made multiple relatively small changes to the client code base over the last week which has resulted in us now seeing the majority of the client tests passing consistently in a multi-section network.
- With the client stabilising, we’ve been able to re-enable reward payouts, and are now working through issues and optimisations.
- The implementation of lazy messaging continues across the board.
- More questions roll in to the AMA, and another answer too see @jimcollinson’s latest video here
- We posted a feature summary for the upcoming testnet earlier in the week.
-
Keep a regular eye on the Like This Tweet thread on the forum for some excellent guidance on how to help promote the Safe Network, and surrounding components, with a simple button click!
BambooGarden Fund Update
A big thank you to each of the volunteers for the BambooGarden Fund committee. We’ve had 8 volunteers offer their time and so we feel this is a good point to close the committee, for now, and keep it at a manageable number.
We were initially intending to have 3 community committee members at any one time, but decided to change this to have all 8 of these volunteers on the committee. This change has several benefits, such as allowing us to make the most of the varied experience and expertise they offer. It’s also inevitable that there will be times when some committee members have other commitments, so this larger number means we can continue seamlessly regardless. Members will be able to abstain when they can’t make it, or even if they don’t think they’re qualified to judge an application, without having to be concerned that they will be leaving the committee short.
MaidSafe staff will still make up around 3 members of the committee. We’ve also added a condition that MaidSafe will have the right to a “veto” on any applications for funding, i.e. the right to reject proposals that we believe don’t meet the objectives, fundamentals and values of MaidSafe, or that don’t meet the original purpose and goals that this fund was created to achieve, and which MaidSafe has been asked to act as guarantors of. Note that a veto is only for rejecting applications, not for forcing them through when the wider committee has rejected them.
All this has been communicated out to the committee volunteers over the last couple of days, along with some other important decisions. For example, the committee will all be members of a separate discourse forum, where we will be able to discuss and vote on applications. MaidSafe have also proposed some immediate areas which we have identified as urgent in taking steps to meet the number one purpose of this fund, which is “to enhance the pace of deployment of the Safe Network”. These areas are:
- Formal documentation - we believe the project would benefit hugely from having our algorithms formally documented. Perhaps a technical writer or the like who has experience writing formal algorithms and papers would be able to offer value here. Having these formal documents in place helps onboard developers who are getting involved, while also helping external auditors who we would at some point be asking to check our code for security flaws, bugs, etc.
- Additional CRDT expertise to help push this to the next level.
- Additional consensus expertise.
- Additional networking expertise.
If you have ideas for fund applications in these areas then don’t send them in just yet, but you can start to think them through a bit more in preparation for the application process to open.
We are also proposing that, once the testnet is live, we ask for applications to generally increase our engineering team size so we can move at a quicker pace. The reason why we say after testnet here is that the onboarding of new engineers beforehand would be a distraction from the testnet work.
Over the next few days the committee volunteers will all be invited to join a separate discourse forum where we will formalise some of the committee guidelines together. We will then announce that we are ready to accept our first funding applications, with full details of the specific initial areas, such as those outlined above, that we want the first batch of applications to be restricted to. We will also provide full details on how to submit applications, and what we expect applications to consist of.
Safe Client, Nodes, Routing and qp2p
Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan
Safe Routing Project Plan
Client code enhancements
After getting the new section wallet split changes merged across the repos at the start of the week, we’ve been doing various enhancements to the client code. There has been some small refactors to simplify things, as well as updates to allow for more flexibility with Elder numbers (as they’re increasing with supermajority changes), but also to use this same supermajority calculation for response handling (i.e. do we get the same response to a query from X Elders and can therefore trust it). We have also added a few fixes for issues that prevented the client from bootstrapping to the right section in a multi-section network. With this in place we were able to see the majority of the client tests passing consistently in a multi-section network.
Rewards
After the above mentioned merges, we re-enabled reward payouts and have been inspecting the behaviour over several splits. A couple of small issues were found and addressed, and a couple of others are still being worked on where there’s excessive CPU and memory consumption after a few splits.
Even though the reward payouts do work now, it looks like we could simplify them even more, and so we will be trying to cut away a big piece of logic handling the payout at node relocation, and instead include the reward payouts at the section wallet split.
Lazy messaging
Alongside this, we’ve updated the lazy messaging implementations across the libraries, and have sn_node adapted to the updated messaging code. Updating sn_routing for these changes has been a big task as every routing message will now require a Header of its own that furnishes the destination’s details, which will be verified by the receiver for correctness and used to update the sender or receiver with the current Network status, if they are lagging behind. Core implementations are in place and we’re updating the last tests there. Once we have this in, we’ll be free to create the lazy messaging pattern in sn_node to enable easy handling of bringing nodes up to speed.
Routing neighbours
In routing we removed the neighbours concept. To recall, nodes need to keep information about other sections in the network so they know (among other things) how to send messages to them, and so on. We used to keep only those sections that were “neighbours” of our section. This was defined arbitrarily as those sections whose prefix differs from our prefix by exactly one bit. This meant that if we wanted to send a message to a section that was not our neighbour, we had to relay it through our neighbours. We realised a long time ago that this is unnecessary complexity and we decided to remove it and this week we finally got round to it. So now nodes keep info about all the sections in the network, which means there is no need to relay anything as they always know the final recipients. This simplifies the design and improves performance. One might worry whether this requires a huge amount of memory but it turns out that it doesn’t. We calculated that we can support several hundreds of thousands of sections before memory becomes an issue. We’ll deal with that problem when we get there. As part of this change, we also refactored the lazy messaging logic and extracted it into a separate module, which allowed us to add more unit tests to it, increasing the confidence in the code.
API and CLI
After finalising the changes required for the new Register CRDT data type in the sn_node and sn_client crates, we started adapting the sn_api codebase and its API to support it.
The main aspect of these changes are related to the fact that we would now be exposing to the user of the API the possibility of encountering branches in the data. These branches could have been caused unintentionally by the application when multiple instances are concurrently writing values to the same Register, but they could very well be created intentionally by the application, for example a blockchain application where forks simply happen and are also resolved.
We also started with adapting all our abstractions (NRS, FilesContainers and Wallets) to make use of this data type. This is where we see a bigger challenge in terms of UX, given that reading a FilesContainer could result in seeing more than one state as explained above. Thus, we are now looking at the best way to expose this in our API so the user and/or developer of an application can decide what to do in those situations. For example, the end user may be prompted to decide not only which branch to read, but also perhaps how to merge them into a single branch again. As you can already see there may be several ways the user or developer would like to resolve the occurrence of branches in the data, and this is the main driver now for the new design of these APIs.
When it comes to the Safe CLI, we’ll need to make some decisions on what options to offer to the end user for resolving data branches. An example here being when fetching a FilesContainer with two unresolved states, the CLI can probably present detailed information about them and let the user decide which one to look up.
CRDT
We’ve been continuing research on the Bounded Counter work, and this week we’ve been working to make them Byzantine Fault Tolerant and to implement a PoC. Hopefully in the next week or so we’ll have something concrete to share on this front.
We’ve also fleshed out the design for a new MultiMap CRDT based on the MerkleReg. This will likely be the basis for NRS subdomain data types, as well as a flexible structure for other applications that need a map like data type.
Community & Marketing
More questions roll in to the AMA, and another answer too. This time @jimcollinson tackles one from @dimitar: Is it too late for online privacy? Haven’t they got all our information already?
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French ; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
