Massive thanks to everyone who joined, or tried to join, the Joshnet last week. We really appreciate your enthusiasm, and it has helped us proceed towards the next testnet (naming suggestions please!). First, we were delighted it stayed up for so long. This was completely unexpected seeing as it was basically held together with string and rubber bands, so it’s a great demonstration of the robustness of the libp2p foundation that we are building upon.
For the next testnet we want to use disk storage rather than squeezing everything into memory, which will be a more interesting user experience.
During the Joshnet we noticed a couple of issues that need addressing. One was unexpectedly high bandwidth. We suspect this is due to nodes apparently joining from behind a router (NAT), but not actually being detectable by clients, resulting in a lot of messages going nowhere; if so, this possibly having an effect on other nodes in the network. We’ve also found high node memory and CPU in the nodes, which seems to be due to the way in which libp2p handles data republishing. We’re on that one right now.
Finally, we realise the “onboarding experience” was a little rudimentary (to say the least), so thanks to all those who persevered. Rest assured Chris is on the case to make this much smoother for the next testnet, or if we are quick in fixing the other issues, the one after - we are super-keen to relaunch!
General progress
@roland is taking the lead on saving data to disk in readiness for the next testnet. And @Anselme is porting further data types to use the RecordStore setup from libp2p, so that we can do away with more custom code and rely on their automated republishing.
There are a few bugs with NAT traversal as it is implemented in libp2p and Rust. As a first step, we want to be able to auto detect nodes behind NAT so we can take appropriate measures, e.g hole punching. @Bzee has submitted a couple of issues to the rust-libp2p repo and experimented with AutoNAT so that network nodes can detect whether a joining node is behind a NAT.
In relation to this, @aed900 started on the bootup/discovery process and worked with @bzee on the optimal setup in relation to the role of AutoNAT and the Identify protocol. The Identify protocol is what nodes use to communicate about the observed addresses of other nodes.
@bochaco has been digging into testing, specifically a basic test which can spawn tasks to churn nodes, plus create and query registers all concurrently on a testnet.
@Anselme is working on the DBC refactor, while @chriso is still on the OpenSearch stuff for monitoring, while also putting together the ‘safeup’ script for easy automated testnet launches on a variety of environments. Meanwhile @qi_ma has been fixing bugs, smoothing out the data replication flows and working to reduce node memory and CPU usage too.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
.
