The soak tests that we ran last week had very positive results. We’ve now started running an internal testnet with Mutable Data and we’re trying out the authenticator and the example apps against this internal testnet.
There are still some things we need to implement before releasing a public testnet. While this will be a client-only network, tests still involve churn to deal with special cases where a few droplets may go offline and these are cases where churn is expected to handle the data requirements as we’d expect. This also allows us to add more nodes to the network if there is a requirement to add more usable space to the network. Initial tests have identified a few bugs that did not occur with mock network tests which the front-end and client team are working on right now while the back-end team are also addressing some of the spam prevention requirements in parallel.
After 18 months of working with MaidSafe, Andreas (@AndreasF) has decided to move on. We discussed him leaving, at least for a while, he is more than welcome back in the future and the door is open. We would like to thank him for working under such pressure and achieving so much in doing so. Heading up the routing design team, Andreas has been a committed and well respected member of the company. As you know he was very much involved in the recent changes to the Routing library and we all wish him very well. One of the benefits of the guys working so closely together is that a number of our engineers are very familiar with that part of the system and its future design direction.
Andreas spent a lot of time with Spandan assessing potential new recruits and scaling the team continues to be a focus. We are working with a number of recruitment agencies and are starting to see a larger than ever before volume of CVs. While the front-end team has taken on a couple of new developers in the last few months, and we are close to confirming more, the network team has not grown much and we are making changes to improve that situation with a view to speeding up development and spreading the huge workload much more evenly. We recently dropped the necessity of requiring each candidate to have Rust experience, and are also looking for system level devs (C, C++ and Rust) who also have experience working with other P2P systems, or network security, or other distributed technologies, such as Hadoop, Redis or Cassandra. This means we are looking for system level devs experienced in Rust, or interested in learning it 
SAFE Authenticator & API
We tested the front-end APIs and applications against an internal test network. We came across a few issues which were not present with the mock network. The entire team has been focusing on fixing these issues and we are now almost done fixing them. However, applications don’t work as expected when revoked and reauthorised. We also found a gap in creating the unregistered client and it was wired up in safe-core. The unregistered client related changes were integrated with authenticator and the Node.js API. In general, it is expected that an unregistered client should be created without even needing an account in the network. At present, it is required to have an account to create an unregistered client. This issue will be addressed in the next few days.
Other than the functional issues, we have few a UI updates that we will be working on as we continue to test and fix the issues.
The changes that were made in the client libs must be integrated in the Java API. The test cases are being wired at present for the existing integrated APIs.
SAFE Client Libs & Vault
We keep catching and fixing bugs in SAFE Client Libs and we’re adding missing functions found by the front-end team. As described in the section above, we’ve been working on adding missing functions to propagate Crust bootstrap configurations to unregistered clients. We’ve also fixed account registration and login functions that weren’t following the calling convention that we use for all other functions exported from Rust.
Currently we’re working together with the front-end team to correct the app revocation and reauthentication flow and we’re adding more automated tests to make sure that these features work as intended.
There are some more bugs in mock-routing that were discovered by @shankar, concerning handling of MutableData addressing and storage. The current mock-routing implementation assumes that only one MutableData instance can be stored with a given XorName, while in the real network that is not the case: apps can store multiple MutableDatas with various type tags under the same XorName. Another issue that we’ve found is that network status is not propagated to the front-end properly and it is not properly tested on the Rust side. So this is what we’ll be addressing for the next couple of days.
For spam prevention, we are trying to see what numbers are good enough a compromise. We have also identified more areas that can potentially be patched and some of them are already underway. That for e.g. addresses the problem that a client can send RPCs which it shouldn’t (but does it because it’s malicious). Soon we will be able to come up with a list of approaches and then we’ll present that information in this Dev Forum topic.
On the vaults side, we’ve added tests and checks to forbid duplicate message IDs for client requests. This is required because if clients send two (or more) request using the same message ID, the account balance checks could be bypassed as the balance would have been charged only once. We’ve also added some new tests and increased the test coverage.
Routing
The mutable data branch has finally been merged into master! That’s a huge change, although not quite as big as the one for safe_vault is going to be. We are now internally running test networks and adding the last few pieces to the implementation.
Apart from minor tweaks to Option B, we are now working on extending the simulation in two directions:
- It used to assume for simplicity, and to first only test the convergence aspect of the proposal, that every piece of information — every vote — is sent to every node in the network. We need to work out, test, and simulate the details of who actually needs to send the votes to whom, and how to recover from cases where messages failed to arrive.
- The simulation is too memory-hungry to create a large number of nodes, but there are ways to deduplicate some of the information shared by the nodes (votes and blocks that are held by the simulated nodes) without changing the semantics. And of course the first point will also help: if less information is exchanged, the code will run faster and with less RAM.
And finally, a new option to mock crypto primitives in tests speeds up the test suite more than 4-fold!
.
We are getting there, but have many more long days and nights to go, so we get on with it.