Safe Network Dev Update - September 24, 2020

Development Updates
1

Summary

Here are some of the main things to highlight since the last dev update:

  • We renamed the safe-vault crate to sn_node.
  • In sn_node we’ve re-enabled the CI system for a subset of our tests, giving us all node tests across all platforms, and almost all e2e tests.
  • In Routing we have now resolved all remaining unit test failures from the removal of parsec, leaving only a few failing integration tests to tackle.
  • The Routing PR to expose an async API has now been merged into the master branch. :tada:

5
5830×430 446 KB

CRDT

sn_fs (previously safe_fs)

This week some additional polishing of the pass-through filesystem support was done. Also, a decision was made to call the upcoming crate sn_fs for consistency with other crates that share the “sn_” prefix.

Safe Client Libs, Nodes (previously Vaults) and qp2p

Safe Network Transfers Project Plan
Safe Client Libs Project Plan
Safe Network Node Project Plan

This week we concluded our discussions on whether safe-vault was the best, most accurate name for the crate. These discussions on this crate had actually been ongoing for a few weeks now, with several suggestions being debated. We settled on sn_node as we believe it breaks down exactly what that crate is, in the most simplistic term - it is a node on the Safe Network. Anyone familiar with peer-to-peer networks will understand this technically accurate term and what it means.

So, Nodes this week has seen our client event handling refactored a wee bit and then merged into master. With this in, and after some other small test fixes, we’ve re-enabled the CI system for a subset of our tests, giving us all node tests across all platforms, and end-to-end (e2e) tests going on Ubuntu and Windows. This is great as it’s not only bringing us back up to proper test coverage for more solid PR reviews, but actually it is something we’ve never had running properly against the nodes themselves before as part of CI. Going forward, we’ll be stabilising this across all platforms (currently macOS is being problematic), and enabling client side tests against the nodes as we get those going once more.

Continuing on from the previous week, the last stretch in our pipeline to align all the modules is the integration of Nodes and Clients. As our newly async routing layer becomes more stable, which in turn improves the stability of the network sections, we are also strengthening the upper layers where data and transactions happen. We are off to a good start here with the client creation happening without much tinkering, which reflects that the nodes, together as a section, are communicating and responding as intended within themselves and with the client. As anticipated, bugs are beginning to show up at the data-transaction layer as we push forward increasing the complexity of the tests. We’ll be taking these on one by one until we see all the tests go green and get the network solid and working as it should be. So sit tight with us on this last stint, and we might just see a baby farming network waiting for us!

Routing

Project Plan

In Routing this week, the work to fix some bugs around DKG and relocation, mentioned in last week’s update, was merged, addressing the majority of failing tests. To complement this, there were two further PRs (fix parsec removal refactoring and simulate accumulate dkg_result) which were also merged, resolving all remaining unit test failures, leaving only a few failing integration tests to tackle.

We’ve also now merged the work we’ve been doing on exposing an async API :tada:. Several fixes and enhancements have been made to make this ready in recent days and weeks, and we now already have the async routing API in place and functional, forming a section when used with our node implementation.

The ongoing research in relation to converting routing data to CRDT continues at good pace. There are many ongoing discussions on how to properly make use of CRDT in this context. We have also started coding PoCs of a couple of different approaches to better understand how to properly architect and design it, as well as see the implications of this whole move. So far, in our heads, it is looking good and making sense, but only in working code do we trust :wink:

Safe Network App and UX

Feature Tracker / Screens & Flows

This week we continued the work of integrating the Action Menu feature into the Safe Network App design prototype. You’ll see it beginning to trickle through to the layouts in the Figma file over the coming weeks as the design is refined and iterated upon.

We were also pleased to collaborate with other designers on the Decentralization Off The Shelf design workshops this week.

With the help of teams such as ours, this project aims to build a library of design patterns that solve problems common to the decentralised networks.

While many of the problems discussed will be solved by default in the Safe Network (:smiley:) numerous other projects are tackling similar issues of language, mental models, and introducing unfamiliar paradigms to new users. So it was a great opportunity to pitch in, collaborate, hear of other innovative approaches, and generally tap into the design-hive-mind.

Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

69 Likes
3

First! Now to read!

22 Likes
4

Thanks @Maidsafe!

25 Likes
5

How will this affect use of the term ‘vault’ elsewhere, in the CLI and most of all Safe Network App and user facing terminology? If ‘vault’ is to remain, and I think there’s a good case for that it seems to create an unnecessary discrepancy.

Meanwhile, a quick question. After re-installing Ubuntu and copying back all my files and configs I find that the safe CLI is complaining when I use safe vault, so I’m wondering how I can fix this without breaking my existing installations. All the Safe apps under ~/.safe and ~/.config are restored and I’ve not done any safe update or safe install stuff, so I’d expect safe vault to be fine, but ‘vault’ doesn’t appear under safe --help. How can i get safe vault back so I can test latest vdash?! :fearful:

13 Likes
6

Great work MaidSafe team! It’s good to see the async/await additions and the new sn_fs … 1000% worth the effort and time.

Always look forward to reading the weekly update, but very late here, so now off to bed! :wink: Cheers

16 Likes
7

What is it complain about? if it doesn’t find the CLI binary I guess you just need to set the safe-cli binary path in the PATH env var again in your ~/.bashrc

6 Likes
8

To avoid any confusion, the term vault has/will be replaced everywhere. PRs are merged in just about every repo where that term is used now, including sn_api so this will result in new commands such as $ safe node update, with node being used anywhere vault was previously. Even the vault folder at ~/.safe/vault will be replaced with ~/.safe/node.

This won’t be until we release the next versions of node, cli, authd, etc, so nothing anyone needs to use just yet. We’ll communicate it out at the time to make everyone aware.

16 Likes
9

The naming updates is great and super helpful for getting up to speed as it matches more with terminology one is already used to. Nicely done

13 Likes
10

It’s good to have one term everywhere, but I think vault is much better for users and good enough for developers. Has there been any confusion that needed clearing up here?

Some devs will also see ‘node’ as node JS, so while I get that it is more precise in relation to p2p, I’m not sure that’s a helpful change even for developers, and think its a much less suitable term for farmers or users. ‘Node’ says very little, and blends into every other p2p project out there.

9 Likes
11

My God it’s Thursday already!! Literally had not registered till I saw this… nice surprise!!

I’m hoping somewhere there are real world readable equivalents; so, perhaps “file system” etc that is nice and simple and relatable.

Would be nice to have a sense of the fraction of work done and I know it’s not linear in time but hard to sense atm what’s still to be done, is it just the parsec fix that’s being worked through?

and did lend itself to safe… vault is a place to put valuables.

Still, most important is the nuts and bolts - naming is minor… but consistency is good to see.

:beer:

7 Likes
12

Thanks Gabriel. I restored all my dotfiles and can confirm that the CLI path is set in .bashrc and PATH variable. Here’s what I’m seeing:

$ safe vault run-baby-flemming
error: Found argument 'vault' which wasn't expected, or isn't valid in this context

USAGE:
    safe [FLAGS] [OPTIONS] [SUBCOMMAND]

For more information try --help

So dutifully…

$ safe --help
safe-cli 0.8.1
Interact with the SAFE Network

USAGE:
    safe [FLAGS] [OPTIONS] [SUBCOMMAND]

FLAGS:
    -n, --dry-run    Dry run of command. No data will be written. No coins spent
    -h, --help       Prints help information
        --json       Sets JSON as output serialisation format (alias of '--output json')
    -V, --version    Prints version information

OPTIONS:
        --endpoint <endpoint>     Endpoint of the Authenticator daemon where to send requests to. If not provided,
                                  https://localhost:33000 is assumed
    -o, --output <output-fmt>     Output data serialisation: [json, jsoncompact, yaml]
        --xorurl <xorurl-base>    Base encoding to be used for XOR-URLs generated. Currently supported: base32z
                                  (default), base32 and base64

SUBCOMMANDS:
    auth        Authorise the SAFE CLI and interact with a remote Authenticator daemon
    cat         Read data on the SAFE Network
    config      CLI config settings
    dog         Inspect data on the SAFE Network providing only metadata information about the content
    files       Manage files on the SAFE Network
    help        Prints this message or the help of the given subcommand(s)
    keypair     Generate a key pair without creating and/or storing a SafeKey on the network
    keys        Manage keys on the SAFE Network
    networks    Switch between SAFE networks
    nrs         Manage public names on the SAFE Network
    setup       Perform setup tasks
    update      Update the application to the latest available version
    wallet      Manage wallets on the SAFE Network
    xorurl      Obtain the XOR-URL of data without uploading it to the network
6 Likes
13

I think that’s the issue, a very old version which didn’t support the vault subcommand.

6 Likes
14

Yeah valid points and similar to what we discussed. This only needs to be called node under the hood, it’s the technically accurate term. The UI/UX can take this node wherever we want to go in future, it doesn’t need to be presented as a node to typical end users. Hopefully devs who do come across it see it for what it is, and in the context of the other renamed repositories/crates & the language that we will be using :slight_smile:

16 Likes
15

Ok, thanks. I’m not sure how that happened! Should give me a way forward although I don’t know which version I was using now. I’ll take a look at my backups etc.

Sorted, thank Gabriel and I have bugs to chase! :roll_eyes:

11 Likes
16

Thank you once again!

It really feels like outlines are getting more and more defined - and pieces inside falling to places.

In regards to this:

… I’d like to ask how you see this work in relation to bigger picture and MVP? Is this something that really comes in to play after next testnet, or maybe something that is essential to get next testnet running?

8 Likes
17

Thx for your incredible work Maidsafe devs

Bring it on baby

Great to see Maidsafe collaborate with others on decentralization :+1: :+1: :+1: hopefully it will also attract new people to this project.

Keep up the good work

16 Likes
18

Rarely do I criticise all the excellent work that goes on here (if ever), but… this is probably the first update that leaves me unsatisfied/disappointed. The renaming from ‘safe-vault’ to ‘sn_node’ is a rather terrible naming scheme. Snake case is fine, but ‘sn’ is non-descript and the whole thing gives difficult readability. Why not ‘safe_node’ or better yet ‘safe_vault’? Please change it back to something more reasonable aka ‘safe_vault’ before too much work has been done. Sincerely and respectfully. :frowning::sweat::disappointed:

5 Likes
19

A large part of the reasoning was safe_XXX as crate names can mislead devs who don’t know the safe network. So safe_network_XXX might be slightly better, safe_ is misleading as it reads like a crate that can not fail or has some engineered safety to it.

vault->node has been weeks of debate and discussion, we will not get a universal agreement, but we went for what is the most common thing that means we need to explain less. i.e. Oh a vault is a network node … and so on, but the hope is this is not debated and if it is we are keeping clear of the debate :wink:

11 Likes
20

Respectfully need to disagree on this one. This change moves the outcome farther from the intended goal. You had perfection with the “safe_xxx” prefix. The whole project is branded “SAFE Network : Secure Access For Everyone”. Having a “safe_xxx” prefix is the most obvious short self descriptive moniker there is. It reads well and flows both in out out of the codebase. Seems rather impossible for an unfamiliar dev to glance over and misunderstand unless they are sleeping. Moving to a “sn_xxx” prefix only obfuscates things and makes it harder to understand. The first question will be “what does the sn_ prefix stand for?” It’s also just plain ugly to look at compared to “safe_node” for example. It also diminishes the branding.

That makes sense, is fine and perfectly reasonable. It’s pretty easy to rationalize a safe network vault as being a storage specialization of the more generic safe_node.

It’s not misleading if being a secure Safe Network code is the intended goal, even if there are bugs in practice that need to be worked out. Just like any other GPL security product like Hardened Gentoo or PAX, devs are familiar with the GPL disclaimer.

8 Likes
21

Bits and pieces under the hood don’t need elegant names :expressionless:

That it works and the UI is simple and intuitive, is where it counts; and good progress is evident on both fronts.

4 Likes