Summary
Here are some of the main things to highlight this week:
- We released the first stable MaidSafe.SafeApp NuGet package built using the
safe-apilibrary
- We made a new SAFE Browser Alpha release.
- We made some significant simplifications and corrections to the code in both the Routing and quic-p2p libraries.
- Vault Phase 2 testing continues, with a testnet in the wild edging closer.
Project Boards
This week we decided to remove the 3 shared project boards that we had on GitHub but which were a couple of months out of date (thanks to @Cryptoskeptic and @Antifragile for bringing this to our attention). We track our work each day through each individual repositoryâs project board(s) and although it is undoubtedly useful having shared boards where the progress from each individual board can be aggregated together, in practice it has become a maintenance overhead since we reduced our team size, and ultimately was left unmaintained and therefore misleading. We want to do everything we can to avoid distractions as we power forward. Where applicable, each section in each dev update should have a link to a project board, and we will work to keep those boards up to date to match our written progress updates.
Vaults Phase 2
Weâve continued with Vaults testing and weâve been making some interesting observations. One thing that weâve observed was that the memory usage might be because of the holding of the PARSEC graph in memory. With this in mind, we are looking at different options to optimize the storage of this graph by potentially writing it to disk, or pruning it more often than we do right now. We are analysing various parameters before we take a step in this direction. Another observation that weâve made was that there is a considerable spike in memory usage when a node goes offline. The Routing library attempts to retry sending the message a number of times before declaring it as âFailed to sendâ and weâre looking at possible improvements that could be made here to avoid the memory spike. With these new observations weâre looking at a potential testnet (with certain limitations) so we could test the different components working together out in the wild. This has spurred a lot of internal communication and we are discussing ideas for re-ordering some work to allow us to iterate testnets quicker and with a clearer purpose in each iteration.
SAFE API
We made some minor tweaks/fixes to our CI scripts as there were development builds required for mobile testing which werenât being published on S3. These artifacts are not needed for users, only for mobile development and testing.
After that we invested most of our time analysing how to move forward with safe-api and CLI in terms of how they fit in the next milestones and plans, e.g. Fleming and beyond. We are still trying to finalise the plan but we seem to be moving towards having the new data types implemented end-to-end (E2E), i.e. from vaults to CLI, making sure the same use cases and scenarios still work as they do now with current safe-api and CLI features. Just to clarify, this is in parallel to the efforts and code base for the Fleming release, and we therefore will start to work on separate branches to bring in the new data types implementation in the corresponding repos, including safe-api.
We are also starting to run some E2E tests using the CLI test suite against a single-section network, lending a hand for the testing process already underway for Vaults Phase-2 releases. We are hoping that this can help in the process of troubleshooting and making sure that itâs stable enough for sharing with the community, as well as getting a better picture of how a single-section network is behaving with the CLI E2E tests and use cases weâve been playing with during the last few months.
Labelled Data, Indexing and Token Authorisation
Weâve been firing deeper into tokens this last week, making good headway into storing tokens for reissue (which involves changes across the authenticator client), thus enabling already registered apps to update their token if there have been any permission changes. Weâve also been updating our mock test implementations to start requiring valid tokens from app connections. So right now weâre progressing through the tests, identifying where those tests are lacking token generation or need tweaking.
Data Types Refinement
The work on this RFC keeps going on at a slower pace, in parallel to other tasks with higher priority for Fleming.
Some changes that came up during review of the Sequence PR introduced regressions, that were eventually solved. Now this branch is just waiting for merge. Next up will be converting safe_client_libs to handle the Sequence type, for running more tests. After that itâs safe-api and safe_vault, and when Sequence has been implemented and tested end-to-end, weâll continue with Map in safe-nd.
Discussions in the Data Types RFC forum topic on resource usage for different use cases of Map in Private scope, led to some new ideas proposed by @tfa for solving this.
Additionally, in collaboration with @tfa, a new useful feature implementation for Private Map was modelled, which would allow resetting a key history back to an earlier version.
SAFE Network App
Weâve continued to develop the interactions around setting permissions for individual files, folders, and apps.
Now with the power of Labelled Data, we are able to extend this further, giving the option to users to set separate permissions for what an app can do with its own data â and the data you create with it â and permissions that apps are granted over the rest of your data.
This ring-fencing of data should allow a lighter touch to authorisation for new apps to get up and running, and provide a smoother experience, while still keeping a handle on data security. It has meant reworking some interfaces, and amending the approach for default permissions for new apps too, but worth a little extra investment for much more flexibility.
Along with labels comes the opportunity for âsmart foldersâ too. Not only a way of grouping data within a file management view without a rigid tree structure, but also a way to further enhance permissions as well.
For example, I might want to exclude all location data from sharing or publishing, or be asked to double-check before any photo can be shared, regardless of where these data reside or what app was used to create them.
Thereâs a lot of ways to cut and order this data, which means a nuanced and sometimes challenging flow to design, but good strides have been made and weâll have more pixels for you to peep at soon!
SAFE Browser (desktop)
At the end of last week, we finally fixed a long standing issue in the browser which was blocking @anon57419684. You can now use the browser fetch API to request files directly from SAFE via the standard JavaScript (i.e., not safe) APIs.
You can check this out with the latest SAFE Browser Alpha release which we subsequently released early this week containing this latest fix, along with a safe-nodejs version bump, and various other dependency updates.
SAFE Browser (mobile)
This week we continued with the thorough testing of the mobile Browser app. We were able to fix multiple issues found and make some suggested improvements along the way. For example, the mobile Browser now shows a list of all the files from a files container if an index.html file doesnât exist in the targeted files container - this replicates the behaviour on the desktop Browser and is particularly noticeable on sites where some versions have an index.html, while others donât. The fixes include enabling the user to connect to a different vault after re-authenticating without having to completely close down the browser, and hiding the version change navigation buttons when any error page is displayed on the screen.
We are down to one or 2 final issues here, specifically around compatibility with older versions of Android and iOS which we would ideally like to support where possible. We will continue working on this into next week and see what we can come up with.
SAFE Authenticator (mobile)
This week our testing of the mobile Authenticator app found some inconsistent behavior where apps were retaining the new test coin permissions when revoked and re-authenticated. The issue was traced to the safe_authenticator lib in safe_client_libs and a fix was implemented by the SCL team. As a result, we updated the native libs that the Authenticator app was using to the latest version. We recently resolved another issue found during testing to prevent soft keyboard from overlapping input fields on small screen devices.
As with the SAFE Mobile Browser, we are down to our final couple of issues here. The main issue we want to resolve is also around compatibility with older versions of Android and iOS, so we will continue to work towards finding the best resolution for this.
SAFE App C#
For the last couple of months we have been seeing frequent changes in the safe-api and to keep up with that we had released 3 NuGet Release Candidate packages. Since the APIs are more stable now, this week we released the first stable MaidSafe.SafeApp NuGet package built using the safe-api library . This release contains all the changes from all previous RC packages. To learn more about the changes, please check the complete changelog. Itâll be great to see you developing some cool apps using this new package and if you come across any problems, please open an issue in the GitHub repo.
Safecoin / Farming
This week we made a quick prototype of bounded counter (bcounter) CRDT in an interpreted language to get a deeper understanding of it, and wrote up a set of notes based on that experience.
There is a potential for data loss when a bcounter replica goes offline before syncing its latest changes to peers. In typical data center usage, this is not a big issue because they can make each data center a logical replica with multiple physical nodes for redundancy. For SAFE Network usage, an alternative approach must be taken. Additionally there are some complexities when replicas are added or removed from the group that must be handled. Work on these issues begins.
On the DBC front, a second draft design document was written based on discussions from the first one. A primary idea of this hybrid system is that DBCs could:
- be stored in a userâs wallet and sent around the network like a cashierâs check so only the recipient can deposit it. â or â
- be exported outside the system as a true bearer certificate that anyone can redeem/spend, more like cash or a gift card.
Routing and quic-p2p
This week we have made some significant simplifications and corrections to the code in both libraries. This work is ongoing at some pace as these improvements are pre-release cleanups. The simplifications include:
- Remove requirement of getting certificates from quic-p2p and passing these back. This saves us handling mappings of certificates in Routing and allows code to be removed. This removes state that is not required and will aid restarts and upgrades. Note, that authentication of nodes (vaults) is already handled in routing via cryptographic keys. That means for nodes, SAFE already acts as a valid certificate authority from the perspective of authenticity and non-refutability. SAFE also handles authenticity revocations and does so extremely efficiently. Therefore all we need from TLS 1.3 and quic is secured and encrypted communications, but not authenticity.
- Handling connections in a more robust manner. This gets us away from seeing a connection drop as a failure and instead always try to connect to detect
NotAvailablenodes. This will aid restarts and upgrades soon. - Message unification has allowed us to clean up the message infrastructure significantly which helps security and code review. There is still a little to do here when we clean up the connection handling and responsiveness checks.
- Flattening the network structure to allow sections to communicate directly (via the section Elders). This will reduce hops considerably and will allow faster initial networks where we can introduce hops as the network grows. This also gives us a bit more information should any partition happen on a huge scale (such as a country cutting the Internet at the IP level and so on). The process here allows simpler code and the ability for sections to catch up as messages traverse the network (i.e. sections update in the background).
We are also looking at a robust non-responsive check for nodes. This allows agreement quickly on nodes that are not actively participating (closing an attack vector).
Useful Links
Feel free to send us translations of this Dev Update and weâll list them here:
As an open source project, weâre always looking for feedback, comments and community contributions - so donât be shy, join in and letâs create the SAFE Network together
