SAFE Network Dev Update - December 5, 2019

Development Updates
1

Summary

Here are some of the main things to highlight this week:

MaidSafe Loan Application Update

The response to the MaidSafe loan application has been overwhelming, within 24 hours we saw the initial offering of 10 million coins oversubscribed by more than 24 million!

The offering was doubled to 20 million considering this and the allocation spread over all 41 applicants.

We can confirm 39 transfers have now completed; 1 applicant is no longer able to proceed, and 1 was given additional time due to personal circumstances.

Which means we have received to date circa 19.8 million MAID from our community. The level of support from you is truly humbling, thank you!

Vaults Phase 2

Project plan

Both @qi_ma and @jonhaggblad have started working on Vaults, and have already made some important changes, serving also as a good introduction to the code.

Qi changed Routing to expose its mock quic-p2p facility. This change allowed to remove the redundant mock code within Vault.

Jon and Qi also started on using real Routing in Vault integrations tests, with mock PARSEC as the first step. So far there is an issue with the testing network setup, which we are progressing in resolving. When itā€™s resolved, this feature will greatly simplify tests of different network configurations. With mock PARSEC we can automatically set up a network without following thorough processes required by the real network which are not really necessary for testing. This feature already exists and provides a lot of value in Routing, and weā€™re looking forward to using it in Vaults too!

In parallel, @ustulation and @nbaksalyar continue to work on implementing a new handshake scheme. It is a required step to support clients bootstrapping to a full network comprised of multiple sections and many nodes. Currently, we are implementing tests for the new handshake process.

Last but not least, we released SAFE Vault v0.20.1. It includes some of the recent changes, and you can use it to run a Vault instance locally on your machine. A full list of changes can be found in the changelog.

SAFE API

Project plan

We are very pleased to share our new release of the SAFE APIs and CLI (v0.6.0) today! This release is an interesting milestone as itā€™s the first release of the Authenticator daemon (safe-authd), which means we are now deprecating the safe_auth CLI, and all operations can be performed from a single interface: the SAFE CLI.

You can upgrade to the new version using the $ safe update command if you have a previous version in your system, or otherwise just download it from the release page linked above.

The CLI User Guide was updated with detailed instructions for the use of safe-authd with the CLI. Note that this version of the CLI and authd requires the latest version (v0.20.1) of the SAFE Vault, so make sure you have this new version if you want to run a local vault.

During the last few days, we were able to also include in this release a new feature which allows switching networks/vaults using the CLI. This is very useful for users who run a local vault and also use the shared vault hosted by MaidSafe, allowing to switch back and forth between them. The updated User Guide also includes specific details on how this command works, so please refer to the Networks section of the User Guide to learn about it.

A new version of the safe-nodejs package (v0.6.0) was also published which upgrades its dependency with the safe-api. This new version allows SAFE Node.js applications to send authorisation requests to the Authenticator daemon using its new communication protocol (JSON-RPC over QUIC) instead of the system URI mechanism weā€™ve been using up until now. Although this is all transparent to the application as itā€™s all being taken care of by the same auth_app API.

Defining the Minimum Viable Experience

We often bandy about the term Minimum Viable Product (MVP) when trying to describe what we are working on, and what is left to be done before the inception of the living breathing autonomous network. That day.

But with many strands of work, different teams working on different layers of the Network, this term can be the source of diverging understanding, and confusion.

So letā€™s lay it out here.

What we are talking about when we talk about an MVP are the features required for a functionally complete network that delivers full data retention and delivers on the security promises. This is the network infrastructure that underpins all other functionality.

When we have an MVP, we can have a Network.

To the average user though, itā€™d likely be quite raw, need only a basic interface via the command line, and not necessarily even consider the basics of usability, such as human-readable URLs, as one example.

We need to go further than that if we want the Network to be buoyant enough for a successful launch. For that, we also need to build a Minimum Viable Experience (MVE).

The MVE is made up of features and applications that are necessary for our initial, early adopter user base to be able to practically use the network, taking into account their experience, needs and expectations.

The SAFE Network App is a conduit to this MVE. Itā€™s minimal, lightweight, and attainable, but it encompasses all we need to get the network to stand up, in the most efficient route, and then iterate from there.

So this week weā€™ve been gathering the teams for a series of, quite frankly, vocal nodule inducing hangouts to button down and clear the path to MVE, and what comes after.

Over the coming week or so weā€™ll be sharing this, so we can paint the picture of how it will all hang together, and you can see how weā€™ve cut down to the essentials, identified the stretch-goals, where community apps can help out with some of the desirable features that didnā€™t make the MVE cut, and how this all paves the way to the Network weā€™ve all dreamed of.

Flat Data and Labels

In amongst all our development work and UX explorations, the question of data structures, and the possibility of having a flatter data structure has come up a few times. As such weā€™ve been exploring this idea in the Labelled Data pre-RFC thread.

The idea is similar to how weā€™ve imagined ā€˜containersā€™ for apps previously but increases flexibility while still giving app developers easy ways to discover/manage application data. If it works as intended it should also serve to prevent silo-ing of userā€™s data in application containers (think two different photo-apps being unaware of the otherā€™s photos), while achieving some other much-needed MVE functionality (general indexing of user data).

Thatā€™s the gist anywho. Head over to the forum thread to dig into the detail. Itā€™s a valuable discussion that will help inform the exploration of the idea and allow us to put forward a concrete proposal and RFC.

SAFE Client Libs

Project plan

As of PR #1102, rust_sodium is now fully deprecated and the SAFE Client Libs are now void of direct C dependencies :tada:. The various cryptographic mechanisms previously used from rust_sodium are now ported to native Rust crates such as threshold_crypto and miscreant. This gives the consumers of SAFE Client Libs a hassle-free experience without the pain of handling C dependencies. This also opens the doors to support of new platforms such as MUSL and wasm which were previously blocked due to this painful (yet quite useful) dependency. With that major chunk of work off the table, the team will now be focussing on the technical aspects of the labelled data proposal and the Phase 2 vault. In parallel to this, the team also prepared and released a new version of the Client Libraries. Note that this release does not have the latest changes in the master branch. The release was done from an older revision that is compatible with the SAFE CLI. So keep an eye out for a new release, one without rust_sodium :wink:.

SAFE Network App (desktop)

Project plan

Weā€™re happy to be sharing our first beta channel release! As a beta, itā€™s more stable than our alpha releases (which some folk may have noted, there have been many ofā€¦), and it fixes a few issues reported by the community on Windows, as well as some other bumps that we discovered in the release/update process for both the SAFE Network App and the browser.

As such, if you want to stay on the beta channel (and only see the more stable beta releases), you should download that from the GitHub release page. Otherwise, if you want to be on the cutting edge with alpha releases, you can grab alpha.20. You will need to download it again though, Iā€™m afraid. Due to the aforementioned auto-update issues.

Itā€™s been quite pleasant to be able to thoroughly test the update functionality of the SAFE Network App and be more confident in our release processes due to the update channels, so thatā€™s great news. Weā€™ve also further improved workflow using GitHub Actions and will continue to do so.

The only other notable things from this release are that the SAFE Network App now will have a channel named executable (e.g. SAFE Network App Alpha) and that the channel version will work with the same channel of apps. So SAFE Network App Alpha, will download/manage SAFE Browser Alpha. This seems to make sense and helps remove some elements of doubt when working across release channels.

It should be noted for Ubuntu users, weā€™ve seen issues around libsodium linking with the latest 19.10 update. Thereā€™s no fix coming from our side for this as libsodium is being removed across the board soon! Which means this will hopefully be the last release to have these issues (there are similar things previously noted for macOS Catalina). But in the meantime, you should just need to ensure some version of libsodium is available at the location indicated if you come across this issue.

SAFE Network App (mobile)

Project plan

This week we targeted some of the issues from the project board and added a few new screens to the app for the corresponding tasks. The newly added pages include the login page and the create account options page. We updated the app to provide basic navigation functionality and integrated all the existing pages to the navigation flow.

Currently, we are focusing on adding new pages and implementing their layout along with all the required controls for a screen to work properly. The styling and user interaction related tasks/issues will be worked on later.

SAFE Browser (desktop)

The browser too has an update with the new safe-nodejs package. SAFE Browser v0.15.4-beta.1 can be downloaded here. Alongside the API updates, this release has the same workflow improvements as the SAFE Network App, in terms of GitHub Actions, as well as some minor dependency updates and other CI changes.

There is also an alpha release available here.

SAFE Browser (mobile)

Project plan

We updated the mobile browser to support Android 10 and refactored some of the native Android code to use the new platform APIs. We will be updating the iOS support to the latest iOS version next week. We updated multiple outdated dependencies as well for better performance and less bugs. Once all the changes are done, weā€™ll be testing a complete app to ensure the proper working.

SAFE Authenticator (mobile)

Project plan

We updated the authenticator app to use the latest safe_authenticator native libs from the SAFE Client Libs. We also removed the bindings code for all the deprecated APIs.

The mobile authenticator app now shows the test coins permissions in the authentication request dialog. The same changes need to be reflected on the app information page. We will work on those changes next week.

We also made some minor changes in the vault connection file manager option to make it more user-friendly and easy to use.

Node Ageing

Project plan

The work on promotions (adults to elders) and demotions (elders to adults) (PR 1929) has been progressing steadily. We identified and fixed a number of issues thanks to our extensive test suite. There seem to be just two major outstanding issues before this work can be merged to the fleming branch.

The first one is about a node trying to join a section thatā€™s undergoing a split. The join request messages can end up in two different sibling subsections, thus the sections do not reach consensus. We have already a solution to this and it is currently being finalized.

The second problem is that when a node gets relocated, not every other node realises it immediately (due to the eventually consistent nature of the network). So some nodes might still think it is a member of the original section and expect it to participate in the message delivery as such. We donā€™t yet have a clear solution to this problem but several have been proposed and discussed and we seem to be on the right track to cracking it.

BLS

Project plan

Aside from some progress on signature combination being correctly handled, weā€™ve paused work on these issues and PRs to focus on Node Ageing and Vault until (PR 1929) can be merged.

Useful Links

Feel free to send us translations of this Dev Update and weā€™ll list them here:

As an open source project, weā€™re always looking for feedback, comments and community contributions - so donā€™t be shy, join in and letā€™s create the SAFE Network together :tada:

83 Likes
3

1st ā€¦ finally!!! :partying_face:

25 Likes
4

Not so close second, but hey, it feels as good as reading another update and knowing weā€™re closer again.

17 Likes
5

Great progress!

Can the flat data structures work together with Solid? In that way you really create a way to make it easy for two separate apps to understand each other data and help users to be stuck to a certain app because their data is stored within this file format.

14 Likes
6

Great update.

Looking forward to seeing what has been defined as the Minimum Viable Experience. Feels like a clear view of whatā€™s needed for launch is developing.

Keep up the amazing work!

14 Likes
7

Glad to see smooth progress with 99% of 20M loan with about 500k MAID of each contributor to MaidSafe fund.

Anyway nice update.

14 Likes
9

great update as allways :slight_smile:

so MVE is bigger than MVP. But what about Fleming? Where does it fit in?
Fleming = MVE? or more like MVP?

10 Likes
10

From my understanding, Fleming is more of a MVP. It looks like they already have MVE on the way quickly after the MVP happens, from my understanding (via the App, and whatever else they have).

12 Likes
11

More exactly 199% of 10M

6 Likes
12

Great progress across the board! Looking forward to Fleming coming together!

What a fantastic community to fund progress so quickly too. Those close to the project realise the importance of the work, the quality of the team and the steady progress on Fleming. Keep it up chaps!

14 Likes
13

Great update as usual. Thanks so much to the entire team for all of your hard work!

12 Likes
14

Thank you for taking the time to answer my question.

Great update all round as is the norm now :grin:
December always sucks as I know we will all soon have update withdrawal :sob:

10 Likes
15

Thanks for the hard work everyone! These seem to be taking longer and longer to readā€¦

14 Likes
16

Itā€™s always looking some release like mvp or mve take longer and longer

1 Like
17

A post was merged into an existing topic: MaidSafeCoin (MAID) - Price & Trading topic

18

Welcome to software development on a rebuild of the whole internet from the ground up. :slight_smile:
Iā€™m grateful for all the hard work from the Team and Community.

7 Likes
19

Seems to be a lot confusion with MVP vs. MVE. In my opinion MVP is definitely Flemming or you name it. Itā€™s the infrastructure. So thatā€™s a prerequisite for MVE. Imagine you have streets but no gas stations etc. (I know itā€™s pretty simplified you could count gas station as infrastructure too). Would be pretty boring, no? Nobody would use your roads because itā€™s useless if you get stuck in the middle of your trip. Maybe some trucks with own supply could find this useful :).
Anyway I think such things needs clarification. Itā€™s devs explaining something without marketing. I think it does more harm then good. At least me is getting overwhelmed with all those buzzwords. Phase 2, Phase 2.a, Phase 2.a.IV, Flemming, Vaults, MVP, MVE. Ever changing roadmap. Feels like free fall.

1 Like
20

So, there are multiple strands that run in parallel here, itā€™s not all sequential. But if you think of the MVP (roughly) being the infrastructure of the network, and the MVE being built on top, but encompassing the frontend and complete UX stack.

So if we just wanted MVP and no more then that would really be Maxwell (Fleming + the ability to upgrade it), but MVE is what we think is the minimum required a full launch. Weā€™ll likely hit those requirements ahead of Maxwell, but and keep on powering through iterations and pulling in features, so weā€™ll have more than the minimum for launch time. But the game plan revolves around getting that MVE locked in, and then plowing on from there through the UX roadmap.

Not to be cheeky, but if you come here to read the weekly dev update, donā€™t be surprised if you get developers updating you on what they have been working on over the past week. :grin:

There is a tension here: on the one hand we have a demand for all the detailā€”exactly what needs to be done before the Network will be alive and kickingā€”and on the other hand there is the desire for us to just get our heads down and not look up until itā€™s finished and the dream is realised.

Part of striking that balance involves cool-headed decision making on what to work on, in what order, and how deep. So we do need to break things down, coalesce around a shared understanding, and press onward. And that has to happen on a regular basis, correcting course as when required, but most importantly and keeping us all pointing in the same direction, with a single-minded understanding of the job at hand.

So the plan hasnā€™t really changed, but thatā€™s why we have to keep on looking at these things when they come up, naming them, defining them, and moving on together.

21 Likes
21

Chaos Theory on a rather refined level.

1 Like
22

Not to be cheeky, but if you come here to read the weekly dev update, donā€™t be surprised if you get developers updating you on what they have been working on over the past week. :grin:

Absolutely. Point taken. And Iā€™am fully aware of this. (Must feel really hard to be in Davids position to manage this balance)

So the plan hasnā€™t really changed, but thatā€™s why we have to keep on looking at these things when they come up, naming them, defining them, and moving on together.

I know that the grand plan hasnā€™t changed. Just wanted to verbalize my feelings. Dev updates feels like reviewing merge requests with endless of discussion/comments. Not getting it merged while the master is shifting away.

(btw. iā€™am rather a fan of feature flags ā†’ but this has nothing to do with the above)