SAFE Network Dev Update - November 7, 2019

Summary

Here are some of the main things to highlight this week:

  • We will be bringing the Alpha 2 network offline this coming Monday, November 11.
  • We updated the FFI building script to generate bindings for the C language (in addition to C# bindings).
  • We switched to always make use of self-encryption for all files published using the safe-api and the CLI. This will be part of the next release of SAFE CLI.
  • We started the implementation of named containers in SAFE Client Libs.

Restructure

As we are all aware the company is going through a restructure. There are many changes in a short space of time. This process has been very successful and the team, as usual, have been tremendous. Many of the team are still in place, working hard for all of us, and will be for the next few weeks. Everyone is dedicated to making the project a success. That includes those working their notice periods. True heroes and we owe them. The company after this process is complete will be:

David Irvine - CEO
Stephen Coyle - QA
Sharon Tannahill - Finance and admin
Spandan Sharma - Engineering
Jim Collinson - User experience designer
Nikita Baksalyar - Engineering
Lionel Faber - Engineering
Adam Ciganec - Engineering
Josh Wilson - Engineering
Edward Holst - Engineering
Yogeshwar Murugan - Engineering
Jon Haggblad - Engineering
Jean-Philippe Dufraigne - Engineering
Marcin Swieczkowski - Engineering
Gabriel Viganotti - Engineering
Ravinder Jangra - Engineering
Francis Brunelle - Community liaison

With the loan agreement providing funding to ensure this team can focus solely on launch then the chance of success has increased significantly. The community are taking up the reigns on many more activities and Ceilidh has been a star helping out there. A typical example of the reason we all need to work and ensure all of these people are rewarded when we launch. Chris, Calum, Victoria, Lindsay and everyone else have been so helpful and understanding that it makes our desire to succeed for them even stronger.

We look forward to increased Engineering focus and the impact that will have on our progress, which we feel has already benefited from the recent months of effort to increase direction and focus.

None of this would have been possible without a great team in MaidSafe and an amazing community. We are humbled and excited to see what the near term holds for us all.

Alpha 2 Network Take Down

Thank you to everyone who participated in this ā€œShould we kill the Alpha-II networkā€ forum post vote. The result of the vote has been a unanimous decision to take it offline, therefore we will be killing off the Alpha 2 network this coming Monday, November 11.

Please make sure you have taken a copy of any data you need from the Alpha 2 network before Monday, November 11, once the network is down all data will be erased and will not be retrievable.

Alpha 2 has more than served its purpose over the last 2 years. It has been invaluable in helping to onboard people to our vision and for proving previously theoretical concepts, but as time has gone on it has become increasingly outdated.

The Shared Vault is of course still available, see the forum post here for further details.

Marketing

Weā€™re winding down all our internal marketing activities but that doesnā€™t mean weā€™re stopping entirely. The team have been helping out the community by providing insight, background and advice on how to move things forward. There is an epic community marketing team forming and we canā€™t wait to see what happens next. As this will be the last ā€˜officialā€™ marketing update, we just all want to say thank you so much for the continued support, the feedback (the bad as well as the good :wink:) and for carrying on our efforts to new levels. See you all on the forum :wave:

Vaults Phase 2

Project plan

We are continuing with the effort to run the full suite of SAFE Client Libs tests with multiple Vaults. As we started approaching this problem, we have also been thinking about the userā€™s experience: we want to make the process of spinning up multiple Vaults as simple and as natural as possible, making it easy for the community to run their own Vaults at home, and also simplifying our testing and QA work. We have started to outline a simple tool to set up multiple nodes in different environments: a single local machine, local area network, or cloud.

We have also fixed one of the limitations that the Vaults had during Phase 1. We didnā€™t support multiple connections from a single client identified by a unique ID, it was caused by some design decisions in the Vaults code, but we thought that we should support this use case during Phase 2 because some users might want to log in to the network from multiple devices at the same time.

SAFE API

Project plan

Along with some minor fixes and enhancements in safe-ffi, we have also updated the FFI building script to generate bindings for the C language (in addition to C# bindings). Thus any developer can already start using them if desired by simply running cargo build --features bindings from the safe-ffi folder. The generated files will be written onto the safe-ffi/bindings/c/safe-api folder.

As recently brought up on the Dev Forum by @mav, we werenā€™t making use of the self-encryption algorithm/mechanism for the safe-api and SAFE CLI which meant all files uploaded were stored as a single immutable data blob on the vault. We now switched this to always make use of self-encryption for all files published using the safe-api and the CLI. This will be part of the next release of SAFE CLI and therefore will invalidate any previous files uploaded with the CLI onto the vault, thus we are considering wiping out all the shared vault data for the next release to simplify things and avoid confusion to people not understanding why some files cannot be fetched with the new version of SAFE CLI. Weā€™ll be notifying the community about this in advance if this ends up being the final decision for the shared vault, this is just a heads up for all of you using the CLI and shared vault to consider.

Integration tests of authd with the SAFE Network App have been going quite well, we spotted some minor issues and enhancements opportunities while doing this, some of which have already been fixed/implemented and are now part of the re-testing effort.

We also made good progress in finalising a first (limited) version of an interactive shell for the SAFE CLI. The interactive shell will be expanded to support many more operations, especially to cover use cases which are not possible to cover with the non-interactive CLI, like the use case of receiving authorisation request notifications from authd. It enables the possibility to also have a state in the shell session, e.g. allowing the user to set a wallet to be used for all operations within that session instead of using the default wallet from the account, or several other use cases we could cover as we move forward in the CLIā€™s development.

We are now just trying to keep stabilising authd while integration with apps happens, while also preparing everything needed for releasing it, e.g. how we package the binary with the CLI and/or the SAFE Network App. And of course we need to add some instructions to our CLI User Guide to explain how to interact with authd and use it for the apps authorisation flow. For those too impatient, you can be sure we are also very eager to have you all start experimenting with it, so you could get a glimpse of how itā€™s shaping out by taking a look at some updates we started working on for the User Guide around authd and the interactive shell.

ā€¦and we just love the very recent announcement of async functions in Rust 1.39, weā€™ll be definitely looking into it for our safe-api.

SAFE Client Libs

Project plan

As of last week, we have some of the team now working on SAFE Client Libs again. And as always, itā€™s been an exciting few days. Weā€™ve been knocking off some issues from our backlog such as changes to handling configuration files, removing some unmaintained dependencies and so on. These changes are internal for maintenance purposes just to keep the library in shape, but it doesnā€™t stop there. Weā€™re also adding support for clients to use different types of keys according to the userā€™s preference and weā€™re introducing named containers as well. In Alpha-2, every account had a number of default containers (pictures, music, videos, etc.) which could be used to store userā€™s data that can be shared across apps. After some discussion, we decided that enforcing the creation of these containers might not be desirable. For example, a user might maintain an account just to manage his safecoin. To overcome this, the default containers are being deprecated and replaced with containers that can be linked with custom names defined by apps. Weā€™ve just started the implementation for this and weā€™ve already received some thoughtful insights. All your thoughts and comments are welcome so feel free to head over to the project board and fire in your thoughts.

SAFE Network App

This week weā€™ve been raising our eyes a little further into the distance, past Fleming and on to launch, while discussing and planning the user experience around App permissions, and giving users control over their personal data.

Weā€™ve said in the past we are looking to build the UX around Data Sharing Controls: putting meaningful and useful controls in front of the user where risks exist, while allowing a seamless and unobstructed flow where they donā€™t.

There are quite a few moving parts to all this ā€” itā€™s a complex problem which warrants diligence ā€” but a bit of care and planning will allow us to set a trajectory for the end-goal experience, while building incrementally, and testing and gathering feedback as we go.

Expect more on this soon as things begin to come together.

Desktop Development

Project plan

Some final bug fixes and tweaks have been put in place for the update process between the SAFE Network App and other desktop apps (i.e. the browser), ensuring that we can have updates in-app, or silently managed via the SAFE Network App itself. Weā€™ve also improved the release flow to enable simpler testing of apps and the update process. All of which means, weā€™re looking forward to getting a new version of the SAFE Network App and the SAFE Browser out soon.

Meanwhile, weā€™ve updated the authenticator implementation to work with the latest auth daemon branch of the API, which after a bit of testing, is working well. Which means we have an ugly yet functional version which can facilitate logging in and managing authentication requests. And, to top off all that, weā€™ve also started implementing the design styling atop this account management branch of SNAPP, so things will hopefully be looking much more palatable soon enough!

Mobile Development

Project plan

We have made considerable progress with the main LaunchPage and the create account page. These two pages are the ones which the user will first come across in the app and will give them a brief introduction to the SAFE Network. The LaunchPage lets the user directly jump in by logging in or lets them explore by trying out the SAFE Browser, learn more about vaults, etc. The create account page briefs the user about what they can do with the SAFE Network.

We are still experimenting with various approaches to implement the designs and have narrowed down most of them. We have also incorporated a few ideas that make it easier for a layman to contribute to the app. Most of the content (textual and otherwise) is read from JSON files by the app and hence anyone can update these and raise PR(s) to improve the content. This will be very helpful in the future when we want to provide multi-language support.

SAFE Browser (desktop)

Project plan

This last week weā€™ve been tweaking the update process to work with the SAFE Network App across all platforms, and enabling more reliable test builds for testing the update procedures.

Otherwise weā€™ve fixed a few smaller pWeb resolution bugs that were reported, and added more tests to catch regressions here.

SAFE App C#

Project plan

This week was very productive when it comes to bug fixes. We fixed many blocker issues which included fixing the ParseAndResolveUrl, Fetch and Inspect APIs on mobile. With these issues resolved, for the first time we have all the tests passing on all supported desktop and mobile platforms :tada:

During the last two weeks, we were facing a strange issue with the Android native libs generated from the safe-api/safe-ffi by the CI where the native libs werenā€™t reflecting the changes made in the code while the libs for all other platforms were working fine. Thanks to @chriso, who figured out the root cause of the problem and resolved the issue.

We also made some changes to our CI setup and now we have one mega pipeline to build and test the APIs on all the supported platforms. Other than a better testing setup, we have now enabled the CI to auto-release a new MaidSafe.SafeApp NuGet package on a version change PR. In addition to releasing the package to NuGet, whenever any changes are made in the master branch, we generate and upload an updated NuGet package to our build server.

Node Ageing

Project plan

Aside from the usual smaller PRs merged related to bug fixes and cleanups, the main items completed this week include

  • Nodes responsiveness are now tracked depending on how much they participate, and nodes that repeatedly donā€™t cast votes for events will eventually be voted offline. We are now working on using that indicator everywhere and adapting it as needed.

  • A new way of handling dynamic membership was merged, add and remove elder events were removed and instead section elder membership is handled the same way as splits. That is, by creating a new Parsec instance. This drastically simplifies multi-changes like the one needed when Elder size is fixed and Add and Remove come in pairs. It also supports the work on DKG allowing the previous set of elder to control everything and then switch to the new set in a single operation.

  • As a follow up, we now run a DKG everytime we change the Elder in a Section. We do not yet use it, but this will support the work of bringing real BLS for Secure Message Delivery.

Looking ahead there are quite a few PRs in the pipeline for the routing crate, including but not limited to Parsec pruning, Elder promotion/demotion, relocation, performance improvements. And also a few PRs related to the ongoing connection handling simplification. Merging PRs in routing requires strict vetting through soak testing, creating a bit of a bottleneck, but we hope to be able to finish and merge these soon.

Useful Links


Feel free to send us translations of this Dev Update and weā€™ll list them here:

As an open source project, weā€™re always looking for feedback, comments and community contributions - so donā€™t be shy, join in and letā€™s create the SAFE Network together :tada:

74 Likes

First for 3rd timeā€¦

edit:

Some little more testing and it looks like that we have Perpentual web browser.

18 Likes

Second!!! 20 characters

14 Likes

Third for the first time :slight_smile:

16 Likes

100 votes and 100% agreementā€¦ ask an obvious question :smiley:

:point_up::point_up::point_up:

12 Likes

Nice.

Thank you to all the team members who now embark on a new journey.

12 Likes

why is @bart not included in the above list, is he leaving, same about @qi_ma ?

Not sure what I will be able to comment on in these dev updates going forward. The marketing part was the only section I understood :stuck_out_tongue: Well there is one thing I will understandā€¦ Fleming release date. When that is approaching I really hope the community can get the word out!

So you are not happy when we post a list of exact information? Who is on the list is who is moving forward. All we lose are important people. I am afraid I wonā€™t conduct personal staff discussions with you chap.

However keep chipping away man, if you know the staff better than me then your welcome to show us all how itā€™s done. Start a company to develop SAFE and staff it how you wish, however MaidSafe staff are important to us and their personal situations are not your business.

What you have is clear and plain. Now if you want to cause malice and harm to folks personal life you need to take a close look at yourself. Do you not think people are hurt, disappointed, had their lives changed, is that not enough for you? Do you really need to pick bones?

14 Likes

giphy

11 Likes

sorry but not sure why you get angry, just asked if they are leaving or not. if you want launch asap why cut the routing team by 40%, that doesnt make sense.

so i guess it doesnt bother you that the routing team is again getting smaller. cool.

Now you are falsifying numbers, I am not angry, I just find your posts pathetic to be honest.

7 Likes

No what makes me want to bang my head against the wall is your constant negativity.

9 Likes

There is no routing team at all now, wow scary Mary! We are product focused, there is a vault team though. Your insider information is failing you here man, pretty screwed up and again ruin a perfectly good update with such utter FUD.

15 Likes

its no fud , just asked if 2 devs who have been working on parsec and routing for few years now are leaving i guess yes as they are not on the list. thanks anyway

1 Like

:wink:

6 Likes

Wow, this should make things a lot easier for you guys!

10 Likes

Thanks so much to all the team for all your hard work! Sad to see so many great contributors go. Letā€™s cheer on the ones who remain.

15 Likes

Should make the Guinness book.

5 Likes