SAFE Network Dev Update - November 21, 2019

[maidsafe]

Summary

Here are some of the main things to highlight this week:

• We released a new Alpha version of the SAFE Network App.
• @JimCollinson posted some ideas on how app and data permissions could be exposed to end users (see the SAFE Network App UX section below).
• We removed the BLS emulation and are now using real BLS only within Routing.
• Thanks to community member @danda for implementing a new feature in SAFE CLI which allows users to retrieve the content of any file in the form of a hex dump

Community Meetups

Today there are two community meetups taking place.

The first meetup is in Malmo, Sweden, and should be underway right now! This was organised by our very own @oetyng.

The second meetup is in Brighton, England, and @dirvine will be connecting in to give the attendees an update on the latest news at MaidSafe and to answer questions.

Should a recording of either meetup become available we will share it with you.

Remember to keep an eye on the upcoming events forum page for details on all upcoming SAFE Network events.

Vaults Phase 2

Project plan

This week, we are continuing to design and implement the client bootstrap & join process. With the new bootstrap flow, clients will try to find a section that manages them by asking Vaults. If a chosen Vault is not a client’s manager, it will return the connection information for a section that is closest to the client’s managing section, and the client will repeat the bootstrap process until it reaches the target section. This is also how the nodes bootstrap process works, so we are going to reuse some of the internal Routing functions to support this process on the Vaults side.

As you might have noticed, this is the first important step towards supporting multiple sections, which brings us closer to the real network conditions. We decided that in general it’s better to implement the network features with as few approximations as possible because some problems might not be apparent in the simplified environments (as e.g. in the case of a single section network). And while with this approach we might not have a lot of exciting demos as we move along, in the long run it will help us to release Fleming sooner.

SAFE CLI

Project plan

Since we are getting very close to a new release of safe-api and safe-cli we invested some effort in putting up a README at the root of the safe-api repo, moving the CLI User Guide into the safe-cli folder, and also adding a README for safe-authd. We included some diagrams which will hopefully give a better idea of how these components all fit together in the ecosystem of SAFE applications on the client-side of the Network.

We worked on a small refactor on the CLI interactive shell. With this refactor we are now able to support all commands that are normally supported from the command line, with the same syntax and options, but from the interactive shell. This will ensure that the commands are consistent in both the command line and the interactive shell since internally the operations are performed by the same code.

We’ve also been very pleased to see a new feature being implemented by GitHub user @dan-da (thanks a lot @danda!!!), which allows users to retrieve the content of any file in the form of a hex dump using cat --hexdump. The corresponding instructions were also added to the User Guide if you are interested in understanding more about this new feature.

Lastly, we’ve also started implementing JSON-RPC for the communication protocol of safe-authd. We needed a simple format to be able to send some structures over QUIC, e.g. all the information of authorisation requests, or the list of permissions granted to each authorised application, and JSON-RPC makes it really simple to serialise these types of data structures with minimum overhead. We are aiming at supporting the JSON-RPC v2.0 spec. We made really good progress with this in the last couple of days, so we will be able to make this also part of the first release of safe-authd.

SAFE Client Libs

Project plan

SAFE Client Libs has had its fair share of action this week. We felt that setting up and maintaining multiple CI/CD platforms was a big overhead for the team as they tend to break from time to time. We then decided to give Github’s new in-house CI platform GitHub Actions a try which then turned out to be pretty fruitful. It’s more tightly integrated with GitHub, is more flexible, and is quite fast despite us not setting up caching yet. A big shout out to @chriso, @marcin and @StephenC for working on getting GH Actions up and running for SCL, safe-nd and safe-api.

We are also in the process of finishing up a large refactor of Client keys. This work will close the following two issues: #1060, #1053 – please see them for more information. This refactor also updates the version of rand we were using, removing much deprecated functionality. The necessary PR in safe-nd has already been reviewed and merged, and the PR in SCL has been approved and is waiting on CI.

We’ve got a major piece of work in line which is the deprecation of the Rust Sodium crate. To give a bit of background here, Rust Sodium is a crypto utilities crate holding most of the encryption algorithms that we make use of in SAFE Client Libs and Self Encryption. The reason for this deprecation is that Rust Sodium makes use of C dependencies, which is a bit problematic when setting up CI/CD for the crates that use Rust Sodium. Therefore we planned to migrate away from this crate in our repositories and fill in the void with appropriate crates. Issues have been raised and the work has begun to rip Rust Sodium out.

With the successful merge of PR#1069, the deprecation of MaidSafe Utilities is now complete. A minor change to the config file handling APIs had to be done as a side effect of this. The new APIs provided are set_config_dir_path and config_dir. They are basically getter and setter functions for the path where the config files (safe_core.config, vault_connection_info.config, log.toml, etc.) are expected to be.

A minor fix will be going into safe_authenticator with PR#1076. This now allows re-authenticating applications to request for new container permissions and update them accordingly in the user’s access container.

SAFE Network App UX

The past few sprints have involved us focusing our attention on app and data permissions: how a user will exert control over access to their personal data, their privacy, and their spending. It’s not simply about the SAFE Network App—it will involve engineering the architecture of these controls at a network level—but digging into how these levers could be exposed to end users here has helped us to explore the space as a whole.

Degrees of Intervention

In the interests of giving a smooth, understandable experience and placing the App permission controls where the risk is presented, we can think about varying degrees of intervention into the user’s journey, or flow. And it is from these interventions that we can begin to construct the experience.

No Intervention

Authorisation granted without notifying the user.

This would be used for actions that carry low, or no risk to the user’s security, or their Safecoin.

For example, an App could be granted Read and Write access to private data that the user has created via that app, without the user being notified in real time, nor providing up-front consent.

Passive Intervention

A user is alerted or otherwise made aware that an authorisation has been granted, but does not need to take a specific action.

Passive interventions can be used in circumstances where the risk to user security is low, but there is still the possibility for bad or malicious apps to cause inconvenience or moderate expense.

For example, Writing ‘UnpublishedData’ is low risk to the user’s security, but incurs Safecoin cost.

If, for example, there are per-App and Global spending limits, giving a soft, self dismissing alert to the user when a new App begins incurring cost keep the user informed and able to make a direct intervention, without the need to interrupt their flow.

A passive intervention would be expected and cause no alarm if you’ve just hit the edit button in a new app, but if you’re working away on something else, and suddenly a dozen new app notifications flash before you, then you know you have a problem.

Here we can see how passive notifications might manifest themselves. Informative, dismissible, but leading through to actionable control should the user require it.

And here we can see some of the options for these passive notifications.

Explicit, Just In Time Interventions

App usage is blocked until the user takes action.

This is for the actions where the risk is high, such as Publishing or Sharing data, or when an action might incur a significant expense.

A user is interrupted and asked to make a decision before they can proceed.

Here are some screens to give you an idea of how that might work:

Upfront Permissions

Allowing a user to make decisions on what permissions an app has, ahead of time.

Should a user desire it, they can opt to be prompted to check and confirm an app’s permissions the first time before it can be used.

This sort of upfront intervention is unlikely to form part of the experience by default however, as simply giving users lots of choices upfront may appear to give them more control, when in reality it may reduce it, due to ‘pop-up’ fatigue and choice paralysis. We should be aiming for a secure, low friction experience that gives users meaningful choices, so while upfront permissions could be part of the overall suite of tools available, they need to be treated with care in their deployment.

Enabling useful defaults

These controls will be available at an App level, and ultimately right down to a file and folder level, allowing users to genuinely have real agency over their digital lives. It’s quite a lot to deal with, and get your head around at first, so we’ll be taking it step by step. We can also help smooth the user’s journey, and enable them to balance rigid control with a low intervention experience, through offering useful Account level defaults.

Here’s an example of how we can offer up some useful presets at Account onboarding time:

So, all-in-all quite a lot coming together here, covering way more than just the SAFE Network App, but deeper into the Network experience as a whole. There’ll be plenty more to come on this over the next few months, so we’d love to get your thoughts. You know what to do.

SAFE Network App Desktop Development

Project plan

After finalising and testing our new release channel setup, we’ve been testing out an alpha release of the SAFE Network Application. This highlighted some issues with layouts, requests and Windows. Today we’ve gotten some enhancements in place to get Windows going. The issue being Windows needing authorised permission to run the safe auth daemon. As such, it’s looking like there will be authentication confirmations on SAFE Network App start/stop as the application starts/stops (and installs if needed) the background service. That’s unavoidable at this stage, sadly, but at the very least that sees us able to access the authenticator functionality.

A word on release channels:

Going forward we’re hoping to be releasing more frequent updates to our desktop apps, and this means we’ll have three kinds of releases.

There will be the normal releases. These will be most stable and also the most infrequent. If you’re not up for trying out new changes/enhancements or getting involved via GitHub issues, this will be the release channel for you!

Beta releases will be slightly more frequent. Something more like a preview, where there may be issues, but hopefully not. If you want to check out what’s new and don’t mind the occasional technical issue, this should be for you.

Alpha releases will be the most cutting edge, but also most unstable. Here be dragons territory. If you want to help us catch and debug issues, and don’t mind getting your hands dirty in GitHub, then your help here will be most appreciated!

Each release channel will receive auto-updates for that channel, or those ‘above’, i.e., SAFE Browser Alpha will get updated with any new browser alpha releases, as well as beta and stable. Beta apps will only get beta and stable updates, etc.

As things stand, the SAFE Network App doesn’t yet respect the release channel for managed application updates. But we’ll be adding this in shortly (so a SAFE Network App Alpha will only install a SAFE Browser Alpha).

If you want to give the new Alpha version of the latest SAFE Network App, v0.0.5-alpha.2, a try then you can download directly from GitHub here. Any issues you do come across please log as a GitHub issue for us to investigate.

SAFE Browser (desktop)

The new release channel setup discussed above was pioneered using the SAFE Browser this week. We tested internally releasing multiple Alpha and Beta versions, and with a couple of tweaks we were able to confirm that the update behaviour between different Alpha versions, and between Alpha & Beta were as expected. With this in place, we’ll be putting out a new Alpha channel release in the coming days with updated dependencies, including a minor electron bump and safe-nodejs updated to v0.5.1.

SAFE Browser (mobile)

Project plan

We have started to upgrade the mobile browser to support the new Android and iOS OS versions. This includes lots of chore work like updating the libs, removing deprecated API usage code and updating UI logic to work on all devices.

Last week we released the first preview of the MaidSafe.SafeApp NuGet package, and this week we started to update the Browser to use the new Fetch and Inspect APIs. With these changes, we are able to browse websites from local/shared vaults.

The next big milestone on the mobile browser is to enable the pWeb experience. We have already started to implement some of the features and the results are very exciting.

Early preview of under development features:

SAFE Authenticator (mobile)

Project plan

It has been a long time since we released a new version of the mobile Authenticator app. With all the changes in the safe_client_libs APIs and how we are using local/shared vaults for testing apps and websites, we think it’s time to upgrade the Authenticator app as well and provide support for all the new goodness.

So far we have updated the packages used, native libs, the SCL bindings and C# APIs to the latest master commit. We still have to update the UI to reflect these changes which we are targeting for the next week.

In parallel, we are working on a nice feature in the Authenticator app - this feature will provide the users with the option to connect to different local/shared running vaults just by selecting the vault_connection_info.config file . This feature enables users to easily connect to different running vaults for testing and experimental purposes without worrying about any technical aspects.

SAFE App C#

Project plan

This week we made some refactoring changes in the Fetch and Inspect APIs. These APIs were previously returning FilesMap (list of files in a FilesContainer) as a JSON string and so devs weren’t able to use language features like querying on the data without deserialising it into C# types.

After refactoring, now devs get the C# type and they can directly perform any operation on the data without performing any deserialisation. We also updated our tests to ensure these changes are stable.

Node Ageing

Project plan

Work on PARSEC pruning has been merged which prevents an over-sized parsec graph causing any issue. There is progress with handling disconnects via tracking unresponsiveness. We have a couple of options to resolve the bulk polling issue due to the new testing pattern. Testing and implementation work is being undertaken to compare which one would be the most suitable solution.

We are also continuing to progress Promotion of Adults to Elder and Demotion of Elder to Adults. A number of our tests are passing, and we are working on fixing the remaining ones.

BLS

Project plan

We have managed to progress a number of items on the BLS project, in particular with this change, we are removing the BLS emulation and using real BLS only within Routing. There are still some rough edges that we’ll iron out as we progress the remaining tasks.

Useful Links

• SAFE Network Website
• SAFE Network Primer
• Network Fundamentals
• Roadmap

---

Feel free to send us translations of this Dev Update and we’ll list them here:

• Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together

[Zoki]

First 20 charachetrs…

Edit: And what an update!! Took me long enough to scroll down to reply “First”!!

[Mendrit]

Second

Safe Apps release
BLS fast progress
And not much left for Vault 2 and Node Age

Great!

[Future]

3rd!
Nice update, love the ux updates especially those regarding the degrees of intervention!
Good to see that it’s time for a new upgrade of the Safe auth app. There’s so much work on many fronts: vaults, cli, desktop app, browser. This is lit!
Congratulation @danda for contributing at the Safe cli!
Keep up the great work @maidsafe team!

[Nigel]

Urma gurd!!! I saw screenshots!! Time to read

Those screenshots have been under figma lock and key! They look amazing and the mobile preview is very exciting as well. So much to be greatful for in this update and much more to look forward to. Great job everyone @maidsafe!

[Secretariat415]

Thanks to all of the whole Maidsafe team for all of your hard work! Keep the intensity going, there is a lot more work to do.

[upstate]

It’s great seeing community members getting so involved!

Thanks @danda and @oetyng!

Thanks for all the hardwork @maidsafe team!

[andyypants]

Ya I love that people are doing these meetups! Do one in Ontario and I will be there for sure!

These things are probably our most productive marketing right now. If we gonna do it grassroots style then we really need to go build a community.

[betterthantrav]

If an an app is authorized to spend Safecoin, what prevents an app from draining your entire Safecoin wallet?

[digipl]

A wallet App must be authorized to spend safecoin to be able to work. The CLI App too if you want use it to make a transfer.

And, for example, what prevents your bitcoin wallet from stealing your funds?

[JimCollinson]

@betterthantrav

If an an app is authorized to spend Safecoin, what prevents an app from draining your entire Safecoin wallet?

There are a few aspects to this.

First off, what we mean by “spending” here is the app incurring Safecoin costs when writing or editing data. This is as opposed to what we mean by “transferring” Safecoin, which is sending money to another individual, or paying for goods/services.

We possibly do need to take a look at the language here, and refine the terms so they are clearer. Perhaps some testing is in order.

But back to ‘spending’ as defined. There wouldn’t be any benefit to an app dev/hacker designing an app to drain a Wallet, not a financial benefit anyway, through theft, as the Wallet would be paying the Network, not the hacker or a 3rd party.

It could still happen through malice, or poor design though. And that’s when a combination of permissions, controls, and passive notifications come in.

First of all, an app would need to be authorised to spend.

Then there will (not for the initial MVP, but it’s roadmapped) a set of spending thresholds, on a per app basis, and on a global Account level.

And thirdly there are passive notifications, which will alert you to a new app starting to spend. If you’ve just started to use a new app, and clicked the edit button, well then you’d expect to see a notification, and can happily ignore it. But if you are minding your own business and see a flurry of them, then you’d be prompted to take action.

This would be a way of stopping (a likely rare) a series of malicious apps spoofing a legit app, but all coming in under the individual app send limit, but not hitting the global spend threshold.

Passive notifications will be useful for more than just that, but you can see in this circumstance how the combination of all these layers of intervention combine to provide robust security checks, without being needlessly heavy handed.

Hope that makes sense.

[happybeing]

I think a good analogy would be a banking app that could transfer funds, versus an app that’s incurring costs by using up your bandwidth.

For the latter we mostly ignore what individual apps use, though we can dig down into that detail if we want. Instead we might set a monthly total data threshold, with warnings if we’re getting close etc.

I think that works well and would be a good starting point. Monitoring with audit capability is much better than alerts IMO. Less invasive and easier to manage.

[JimCollinson]

@happybeing

I think that works well and would be a good starting point. Monitoring with audit capability is much better than alerts IMO. Less invasive and easier to manage.

I think that’s why passive notifications/alerts should probably default to new apps only, so it’s not too invasive. But there’s plenty of flexibility in there we can play with to get the balance right.

[janriemer]

Wow, again such an amazing update!
Thanks @maidsafe for your continuous effort and such detailed Dev Updates.

@maidsafe

This sort of upfront intervention is unlikely to form part of the experience by default however, as simply giving users lots of choices upfront may appear to give them more control, when in reality it may reduce it, due to ‘pop-up’ fatigue and choice paralysis. We should be aiming for a secure, low friction experience that gives users meaningful choices, so while upfront permissions could be part of the overall suite of tools available, they need to be treated with care in their deployment.

That’s why I you @maidsafe, @JimCollinson. No weird hacker stuff for enthusiasts that nobody knows how to use, except their inventors.
Instead, a coherent, delightful user experience with attention to detail without loosing sight of the big picture!

Also, really nice to see contributions by community members. Thanks @danda for your contribution!

[davidpbrown]

Here for the pictures … not disappointed!

Hoping perhaps those designs with a mind that later they might be skins that a use can add; and also easy switch of languages.

[VaCrunch]

@JimCollinson

We possibly do need to take a look at the language here, and refine the terms so they are clearer. Perhaps some testing is in order.

Maybe an app is either “Safecoin-enabled” or “Safecoin-disabled”. The default would be “disabled” and would require user authentication to be enabled.

[danda]

i’m new here so forgive me if this has been discussed to death before, but I figured i’d give my $.02.

I think many people are not yet comfortable with software agents spending their money autonomously. i certainly am not.

As such, when i read this statement in the update, it immediately caught my eye as something to be wary of:

If, for example, there are per-App and Global spending limits, giving a soft, self dismissing alert to the user when a new App begins incurring cost keep the user informed and able to make a direct intervention, without the need to interrupt their flow.

big/small are relative terms and mean very different things to different people or even the same person at different times. So i think it needs to be introduced clearly and gently. That could mean that the first time an expense comes up (of any size), the user needs to be notified with a modal popop and explicitly authorize the expense and be prompted to set (or review) spending thresholds that fit their particular circumstances.

Then there will (not for the initial MVP, but it’s roadmapped) a set of spending thresholds, on a per app basis, and on a global Account level.

I’m glad spending thresholds are in the roadmap. I would urge you to consider including it in MVP though, at least at the global account level, so that people can have a sense they are in ultimate control of their funds.

[drehb]

@danda

I would urge you to consider including it in MVP though, at least at the global account level, so that people can have a sense they are in ultimate control of their funds.

If this feature isn’t included, a simple work around would be keeping limited funds in the account you use daily, and segregating your other coins into an account reserved for storing coins only.

[neo]

No need for separate accounts as we will be able to have multiple coinBalanes in the one account and just keep limited funds in the default coinBalance

[mav]

Is this safecoin symbol a unicode character? Don’t recall having seen this before. Pretty neat.