Summary
Here are some of the main things to highlight since the last dev update:
- We’ve updated the SAFE Browser with a new beta release.
- We released a new RC MaidSafe.SafeApp NuGet package.
- Work is underway to make the mobile Browser and Authenticator apps compatible with the latest Vault and CLI.
- Message-based voting was successfully merged on Wednesday, and is a major step towards removing PARSEC.
- Last week’s testnet iterations have helped us close in on memory leaks in two places - work is ongoing to find the exact source(s) and fix them.
- We published the new safe-farming repository
This is still an early stage library that we’re using as part of the AT2 changes in vaults.
Vaults Phase 2
Last week we released a couple of testnets with some improvements that we have been working on since testnet iteration 1. The first testnet accepted up to 22 vaults that connected from home. The network received 2956 data chunks from 47 accounts, stored in 414 unique files containers.
After about 14 hours, the Elders of the network were killed by the operating system because of high memory usage. This seemed similar to the issues we identified in the previous testnet, so we restarted the testnet with memory profiling enabled. Thanks to the help of the community, we were able to collect meaningful results that have helped us identify the source of the issue. The reason behind this high memory usage is because of memory leaks in two places. We continue investigating this to identify the exact root cause - at the moment it doesn’t look like the memory leaks are as a result of our code, so we are also looking at some of our dependencies and their usage. We will keep you posted on any advancements on this front.
In parallel, to try to help us find bugs like the above earlier, before they reach the community, we are also looking at some options to help improve and expand our testing environment by automating churn using droplets and/or containers. We are still in the early stages of planning this, so we will share more information in due course as decisions are made.
SAFE Network App UX
A word of warning on the links in this section: there is currently a bug within Figma that is causing high memory usage, so they may become unresponsive, or fail to load entirely.
Feature Tracker / Screens, and User Flows
This week, on top of working through a micro-feature and snagging list too tedious to detail here, we’ve published the screens and flows for Notifications.
These may seem a little underwhelming by themselves, but when used alongside Permissions (and the use rules, and data labelling), are a powerful way of creating a user security model that balances fast and low friction workflows, user control, and permission fatigue.
Passive notification, for example, combined with Default Permissions set to allow a new app only to be used to access data with its own label, means we don’t have to interrupt a user with upfront decisions around what level of capability to grant an app. They can begin to use it, seamlessly, yet have their attention drawn only to actions they have not initiated.
More direct and up-front content models are available of course, with the flexibility that these levers provide. In fact, you’ll also notice that we’ve bundled the options together into two different security ‘profiles’ options that the user can quickly get started with during onboarding.
SAFE Browser (desktop)
We’ve updated the SAFE Browser with a new beta release atop the updated safe-nodejs.
There’s also the incorporation of a more permissive CORS headers here to help with fetches in JavaScript, an update to bring our Electron dependency to v8.4.0, and other smaller updates to the browser’s dependencies.
We managed to fix some broken CI tests in this release, getting our Ubuntu end-to-end tests working once more, which should allow us now to enable automated security updates via Dependabot, once we get these changes into master.
SAFE API
We made a first experimental prototype of the tree CRDT mentioned last week in order to gain some familiarity and test out the concurrent move scenarios presented in the paper. The set-based algorithm is working as described, but is inefficient. Next steps will be to implement a vector clock and implement some optimisations described in the paper.
SAFE App C#
Today we released a new RC MaidSafe.SafeApp NuGet package which includes updated authenticator APIs which are exposed from safe-api/safe-ffi where we previously were using safe-client-libs/safe-authenticator-ffi.
We have already started to refactor the authenticator mobile app to use this package. The authenticator app has its own FFI wrapper, while for the mock APIs safe_app_csharp has a separate copy of the same code. With the new package, we can easily remove the redundant code and reuse the authenticator APIs. The refactoring will also make the app compatible with the latest vaults and CLI.
In parallel, we are working on the mobile browser app so it can be used with the latest vaults and CLI. This will include supporting the updated File APIs which now uses CRDT based sequence data.
CRDT
This last week we’ve been adding some more unit tests to our Sequence implementation, but this time for some specific scenarios in which data mutation operations are sent concurrently with Policy mutation operations to different replicas. With the addition of these new tests, we’ve been able to explore a couple of different alternatives to the solution for these scenarios.
As mentioned in previous dev updates, supporting Access Control on top of CRDT data is a bit tricky and this is why we are exploring alternatives. We’ve been testing two different approaches so far which we don’t believe are optimal, or at least not in the way we defined them so far, so we continue trying to find a better approach and doing the necessary tweaks to either of the existing approaches to make it suit our needs. What we are after is not only supporting these scenarios but also with the simplest implementation possible. The latter is what needs some more effort.
We’ve also made a small refactor in the Policy management implementation in terms of making the owner of the data part of the Policy itself, instead of having separate pieces of content for owner and users permissions linked to the data. This is mainly to avoid having to face another challenge when it comes to supporting concurrent mutations for data, permissions and ownership. Thus we are making the Policy self-contained, holding not only users permissions but also ownership information, which means that changing ownership simply means mutating the Policy.
Transfers
SAFE Transfers Project plan
SAFE Client Libs Project plan
SAFE Vault Project plan
SAFE Farming Project plan
We have mentioned over recent weeks that work was ongoing on the farming aspect of the SAFE Network economy - we can now share this work with you with the publication of the new safe-farming repository This is still an early stage library that we’re using as part of the AT2 changes in vaults, providing some logic for calculating rewards and allowing vaults to utilise this to generate payouts from their section wallet.
In wait for remaining tweaks on messaging, half of the team working on transfers went over to other tasks this week.
Most of the second iteration of the messaging refactor was completed, and with that the last bigger pieces of section payouts (i.e. farming) have also been put in place. This allows us to now go into the finer details of the logic, and to work on the end-to-end testing over these changes as well.
Routing
As mentioned in last week’s dev update, we were working on implementing message-based voting this week. This work was superseded by PR 2161, which was merged on Wednesday, and is a major step towards removing PARSEC. It refactors the consensus events & mechanism into two parts (ordered and unordered) and implements the framework of message-based voting using BLS Signature aggregation, moving the events that do not require ordered consensus away from PARSEC to use BLS.
Also as a requirement from the testnet team, vaults were in need of an event to notify them that they were promoted to an Adult, hence so they can distinguish themselves between Infants and Adults nodes. PR 2162 solves this by firing an event when Infants age into Adults during churns. We’ve also begun work on solidifying the crate by adding more non-mock tests that keep a check on the logic of these fast-moving parts during the overhaul. Up next we’re looking into #2122 which moves intra-section message signature accumulation away from PARSEC.
Useful Links
Feel free to send us translations of this dev update and we’ll list them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together!
for the community also, hang in there super ants.
