Summary
Here are some of the main things to highlight this week:
- We published an interview with Devolution creator @Glen_Simister.
- As we mentioned last week, we now have a good idea of how to replace our concrete coin with a common coin to make PARSEC fully asynchronous. However, we realised that the impact on the Routing crate would add another substantial amount of work. So while we have worked out what should be done and have a clear picture of how to get there, we’ve decided to maintain our strict focus on delivering SAFE Fleming as soon as possible and will therefore document our learnings about the common coin but keep the implementation as a lower priority item for now.
Marketing
Following recent additions to the Marketing Team, we’ve made the decision that we need to dive much deeper into the tech behind Fleming and, ultimately, Maxwell. This is a vital stage as we build up our knowledge, particularly of areas that have evolved in recent months, and refine our messaging in order to ensure that both our content and research is on point. Translated, this means that we’re spending a great deal of our time focused on capturing and recording that knowledge, with the intention of repurposing that information in many ways that will be of benefit to different groups as we move forwards. We’ll have more to say on this over the next few weeks but as it’s time-consuming but vitally important work, we wanted to let you all know. One other thing to point out in the meantime: yesterday we published an interview with Devolution creator @Glen_Simister this week that’s essential reading for anyone who’s looking into building SAFE DApps.
SAFE API & Apps
The planning and scoping for the next safe_app_java milestone has been completed. The code sprint for the same has begun, which includes improving the test suite, generating the coverage report, integrating with coveralls.io and fixing some existing issues. The progress of this milestone can be tracked on the GitHub project board.
The SafeAuthenticator mobile application is undergoing its second development sprint for the UI/UX enhancement milestone. Issues reported by QA in the first sprint are also being addressed. The progress of the milestone can be tracked on the project board.
Regarding the safe_browser, we are continuing to work on some internal improvements in the codebase structure, mainly towards our goal of being able to upgrade to Electron 4, but also as a way to reduce the CI times. Late last week we merged several of these improvements, e.g. we now run all end-to-end tests against the packaged browser app only, avoiding having many CI instances running and install steps in them, both dev and prod packages are built on the same CI instance. We also upgraded several of the dependencies, among other enhancements which reduce the CI times for the safe_browser, making our dev life a bit easier. In addition, we continue tackling down high priority issues to be able to release a new version of the browser soon, based on the original plan we will be creating a first release candidate in the next few days so we can take it to a first internal testing phase.
On some other threads, we are working on documentation for some of our projects, like the browser, safe_app_nodejs and some of the example applications. We understand that current documentation is not good enough for new developers who would like to contribute, therefore we are producing better documentation showing how the codebase is structured, making it easier for devs to locate answers when attempting to implement a new feature or even fix a bug.
SAFE Client Libs
This week we continued working on the documentation internally, creating Confluence pages with Rust development info, contribution instructions, useful info about RDF, Solid, and the Redland libraries, and technical details about the Client trait.
We’re starting to consider how we want to go about making the docs we have public. We’re currently leaning towards migrating the docs to HackMD which has a number of advantages, including a Book mode to structure the pages, the ability to keep pages private but set them to public-readable once they’re ready, and a smoother editing experience than Confluence. HackMD also uses Markdown, which makes it easier to edit than Confluence and also means that we can copy/paste the docs into a Readme file, or even into documentation comments in the Rust code itself. There are other options of course, like hosting the docs on GitHub – it has some of the same advantages, except that there is no way to keep some documents in a repository private while others are public.
Our priority is still the RDF storage proof of concept, which we are working hard on. @yogesh spent some time this week getting up to speed on the Redland libraries while @marcin helped him resolve some issues with building them.
Routing
This week most of the team came back refreshed from their holiday.
On the Fleming front, we continued to refine our Sybil attacks models. We ran more comparisons between large sections composed mostly of elders and large sections with a smaller number of elders. This confirmed what we observed last week: it appears that large sections with a smaller number of elders can be quite resilient to Sybil attacks. We also started modeling network growth and the split of sections that occur as they become too big. The preliminary results suggest only a moderate impact compared to our previous models. This is positive as it means that we can easily extrapolate from the results we already obtained in constant size networks.
As we mentioned last week, we now have a good idea of how to replace our concrete coin with a common coin to make PARSEC fully asynchronous. We started implementing the changes and made good progress. This allowed us to get a good feel for the amount of implementation effort required. The changes to PARSEC are quite manageable: we could mostly implement all that’s needed in a couple of weeks (note that this is before making all the tests pass so another week or two would be expected); however, we realised that the impact on the Routing crate would add another substantial amount of work.
So while we have worked out what should be done and have a clear picture of how to get there, we’ve decided to maintain our strict focus on delivering SAFE Fleming as soon as possible and will therefore document our learnings about the common coin but keep the implementation as a lower priority item for now. This will enable us to come back to the implementation at a time when it is our highest priority item and start with a very good idea of what’s involved. You should expect to see an RFC detailing our lessons learnt and an update to the PARSEC whitepaper in the near future.
We also continued working on performance. We created more relevant benchmarks to support further improvements. In particular, we enabled replaying PARSEC scenarios showcasing exactly how the Routing layer uses PARSEC. This will be crucial to confirm we optimize for real use cases.
A different kind of performance improvement happened this week: the QA team generously helped us improve the CI speed from ~40 mins to ~10 mins. This makes the development workflow that little bit more streamlined, which is much appreciated 
We also finally merged a long-standing performance improvement pull request that reduced the amount of re-processing for a 10-40% speed improvement depending on the section size in our benchmarks.
Crust
We finally finished our work on the bootstrap cache. Externally reachable nodes (ones that have a public IP address) are now stored in an LRU cache. We reused our existing lru_time_cache library. Although, it required a few changes to make it work for Crust and fortunately it will be more flexible as well:
-
LruCache::iter()order was changed to produce the most recent items first. - New methods prefixed with
notify_were introduced that return expired entries that were evicted from the cache.
Now we want to answer the questions of how Crust can help network upgrades, restarting nodes rejoin the network, maximize bootstrap success rate, scale up to thousands of connections, etc. All of that will require Crust API changes which hopefully will make the library simpler too.
I’m assuming the change to the common coin, performance wise would be similar but if for some reason it was not it would be a pain to deal with any unknown unknowns or extensive testing in the face of a public release that is highly anticipated.
found
s