SAFE Network Dev Update - May 2, 2019

Development Updates
1

Summary

Here are some of the main things to highlight this week:

Roadmap

Over the past few weeks, weā€™ve been working hard on pulling together a Roadmap in order to give certainty to the community about our immediate and medium-term priorities. Everyone (internally and externally) is clear about what the ultimate SAFE vision is - but of course there are many different alternative paths available to us in order to arrive at that same destination. Consequently, the discussions have been centred around how we get to the launch of an MVP which (1) does not simply get attacked on release, and (2) does not sacrifice any of the fundamental SAFE characteristics without which the Network would be a pale imitation of the goals that weā€™ve always pursued.

Weā€™re happy to say that in next weekā€™s update, weā€™ll be publishing the updated Roadmap for the SAFE Network. This will be in a format that should bring clarity and allow us collectively to ensure that everyone is on the same page when it comes to the path ahead. As you would expect, it is easier to be clearer about the short term roadmap while medium and longer term features will have less detail and may change. Weā€™ll never be able to satisfy everyone with the details (we wonā€™t be publishing a launch date, for example) but we hope that everyone will find the information useful and a significant improvement on the information today which is captured across many different locations and conversations.

Marketing

This week, we released a new video intro to Sybil Resistance on the SAFE Network by @jonhaggblad. We also released a new post on the topic (on Medium and the Forum).

Weā€™ve also released an update on QUIC, Quinn and Crust
(on Medium and the Forum) plus @ustulation has recorded a video discussing Quinn and the adoption of QUIC P2P. In addition, weā€™ve also released our monthly update for April.

To explain, weā€™ve cross-posted these articles in response to feedback that some are keen to avoid using Medium. However, given the marketing impact this platform still provides us with (automatic emails, viral reach, readership outside the existing community, etc.) we will be continuing to use it - so even if you first read the article on the Forum, please do go onto Medium and give the article support (i.e. claps) as this increases the opportunities for people outside the existing community to learn about the SAFE Network.

Weā€™re considering this further internally because the articles written for Medium are specifically targeted at a more general/beginner audience. Often that content will be simplified and shortened so weā€™re exploring how to ensure that simple cross-posting wonā€™t appear too simplistic on the Forum (where by definition much of the audience has at least a passing knowledge of the Network already). Weā€™ll speak more about this next week.

On the subject of exchanges, a quick reminder to check out the new MAID BTC/ETH/USDT listings on the Bitker Exchange if you havenā€™t already and are trading MAID. Finally, if you use OpenLedger, please ensure you read this update and remove any MAID. And finally, a reminder of the upcoming meet-ups - Glasgow (Tuesday 7th May), SOLID London (Wed 22nd May) and SAFE Network: Brighton (Thurs May 23rd).

SAFE API & Apps

This past Tuesday we released a new version of the SAFE Browser (v0.13.0), which brings in several bug fixes and enhancements such as having a nicer popup for the Authentication notifications feature. You can find more details and instructions to download it in its dedicated forum post, where we provide help for issues faced by users.

We received some really helpful feedback and help with testing the safe_auth CLI from the community (thanks a lot to @tfa and @draw!), with a bug that is now fixed, plus some suggestions for enhancements that we added to the backlog/todo list. We are also trying to apply to this project a similar process that weā€™ve been internally using for the development of the SAFE Browser and incorporate the help of the QA team, and we anticipate the additional ideas and suggestions that will bring will help to advance the project. You are very welcome to look at the safe_auth CLI project board to get more details of upcoming tasks and/or ideas we are throwing at the table.

The Electron project have been releasing several new versions in the past few months which we couldnā€™t keep up pace as we wished, so we are now trying to catch up by aiming at upgrading Electron to v5 in the SAFE Browser, and in our safe_app Electron app boilerplate that is used for the tutorials on the DevHub. We already started working on this in the last few days, and we were and still are facing several issues but we are now very focused on getting it done and hopefully having it ready for the very next release of the browser. We believe this should solve several minor issues we currently have in our browser and therefore itā€™s a good moment to complete the upgrade.

We have however gotten the browser up to Electron v4 successfully, and are building on top of this with a flurry of fixes and features this last week. Including a better, more feature-rich right-click menu, some more wee UI tweaks (thants antd!) and more stable URL-opening with multiple windows. Weā€™ve also been tweaking our workflow, to try and prevent PRs (and all that rebase/merging fun) from stacking up too much. You can read on the repo itself, but may be worth noting for folk here that weā€™re now working on the [dev branch] of the browser, and any/all PRs should be pointed there whereas master will be the latest stable release of the browser from now on.

For those CLI lovers (like some of us :wink: ), we were researching and thinking, for a few weeks now, what a safe CLI may look like, a CLI which allows users to manage all kind of SAFE Network data. We donā€™t have much more to share yet, but we are very excited to at least share that we started working on documenting some of these ideas towards having an RFC for such CLI. We are trying to get this sooner rather than later as we want scrutiny from the community at an early stage, thus stay tuned that weā€™ll be sharing more news on this front hopefully soon.

SAFE Client Libs

Itā€™s been an interesting week for SAFE Client Libs as @lionel.faber and @yogeshwar continue exploring the internals of the Redland RDF API. The Redland C libraries provide APIs to work with RDF graphs and SPARQL queries, so itā€™s important that we understand their design and implementation while writing Rust bindings for them.

In parallel, we are also working towards the improvement of our codebase. We fixed a bug that caused safe_bindgen to generate unnecessary duplicate code and we are improving the crate by fixing a few bugs and adding more tests too.

Weā€™d like to give another shoutout to @riddim and @dask for their impressive work with the Python bindings for SAFE App and a proof-of-concept chat application! If you havenā€™t tried it out yet, visit their post and show some love.

Routing

As per last week, the main thrust of our efforts has been towards getting Routing ready for Fleming.

The refactoring of Routingā€™s state machine has continued and is nearly complete with just PR 1616 still pending. In parallel, the refactoring of the message-relay mechanism has been ongoing, with some discussion on the various options available to us.

Weā€™ve also begun to implement some of the changes needed to handle relocating nodes/node ageing, using the flows described in the routing_model crate for the design. Work on this crate has also progressed, with some cleanups and additional tests being added.

The degree to which Routing abuses PARSEC in its aggressive churn tests has allowed us to find another edge-case bug in PARSEC. Work on resolving the issue is continuing, but this has also caused us to rethink our decision to postpone some of the PARSEC test tasks. Its own test suite should have been sufficient to catch this bug, so weā€™re in the process of implementing some of these backlogged test tasks.

Finally, @jonhaggblad has created a video giving an overview of the Sybil-resistance mechanisms in the SAFE Network.

QUIC P2P

We are continuing to enhance the library and covering more test cases. This week, @ustulation added code to retain messages that were not sent for some reason (e.g., due to a networking error). These messages will be returned to a user of the library and can be attempted to be sent again later. This brings us more robustness and we can be sure we guarantee to not lose any messages users send. As mentioned in the pull request description, this can also be improved further by implementing app-level acknowledgement messages if we would require even more robustness.

We are also continuing to work on adding more tests for the bootstrap process and covering more edge cases and failure scenarios which could happen during peers bootstrapping. Weā€™ve implemented some simple testing utilities to help us with this task: e.g., they allow us to introduce artificial delays for certain connections and writes. It should also help us to cover more test scenarios in the future.

There has been an interesting discussion on the forum about our move to QUIC, and weā€™ve been answering some interesting questions from our community members:

75 Likes
3

First first first 20chars

18 Likes
4

I think the first like should be considered the first one @Zoki, so I win! :smile:

26 Likes
5

Doing my happy dance!!

25 Likes
6

Thanks to the entire team for all of your hard work!

17 Likes
7

Video evidence please.

16 Likes
8

Iā€™m excited to see the new roadmap next week!

I know MaidSafe wouldnā€™t be releasing it if they didnā€™t feel confident about their progress, so this is very encouraging.

22 Likes
9

Damn I just got in the car, you are a minute too late.
Went something like thisā€¦
giphy

24 Likes
10

Thanks to all the team for always pursuing excellence over shiny news.

Instead of being stubborn and keeping crust, you chose to adapt and reemplace it with QUIC.

Instead of going the easy way and launching PARSEC with a bug, you chose to fix it and do more tests.

The price of the token is not happy with your choices right now, its too boring, no shiny news, but the network will be excellent, and the patient shall be rewarded.:heart::heart::heart:

36 Likes
11

Great to hear about all the progress, especially the roadmap! Looking forward to seeing that next week!

13 Likes
12

This is great news I think for lots of people. For me itā€™s not that important. I have the utmost confidence in the team, and the progress of this project. But itā€™ll be great to have a roadmap too!

15 Likes
13

Looking forward to that! If itā€™s possible that is some tasklist that gets ticked off over time, that would help sense of progress over timeā€¦ or even retroactively addin key steps from the past weekly updatesā€¦ so much has been done already and a long list of all that would be nice to see acknowledged.

:+1:

13 Likes
14

Iā€™m obviously biased ( :wink: ), but these recent videos are really, really good.

Brilliant work from the marketing team and guys doing them (@ustulation, @jonhaggblad, and others).

18 Likes
15

They are good enough for me to post on my social media. And I only go for high quality like youngdumbcrypto and madbitcoinsā€¦ no really these are great they make me look like way smarter just for sharing them lol

11 Likes
16

The Roadmap is a double edged sword. Insufficient detail and the speculators will stay on the sidelines. There needs to be a sufficient level of speculation to increase liquidity from current levels. Too much information and you will be held accountable to it again. It will be interesting to see what transpires

Any chance of including Cryptobeadle in the marketing of Fleming?

5 Likes
17

100% it is, folk will take all estimates as promises and extract timelines and wail like sirens as the thing develops. I am keen to just lay it all out and do 2 things, give complete visibility to all stakeholders, Engineers, Marketing and more of what is to be done and what is in critical pat. It will show what need not be done (for launch) that helps us all, that is related to the mvp approach and as @opacey has been (correct) nibbling at will allow us to be very clear and succinct in getting to launch.

So all in the open, what will it really mean?

The plan will identify all components and tasks we see need done. We may miss some, but I am not sure that will be the case. I am prepared we do though and also prepared to rectify it. The plan will be a large set of project, subprojects and individual tasks. We estimate the task, roll up to sub-project (compounding the error we made in the task estimate) and then roll up to the overall project, further compounding any estimate error.

Therefore all in the open == living project, time will change and this is just how projects work, but we will all see that and also see how it is mitigated or just paid for by us. So that is exactly where we will be delayed with a plethora of back seat driver project managers and very loud, ā€œbut this is lateā€ cries stating we are breaking promises as we estimated 1 day and it took 3, we may even be called scammers and charlatans etc.Going into this project phase in the open will mean all of this, but the positives are great, clarity of direction, clarity of critical path are essential for us all, every single member of maidsafe, every investor and every community member can be so much better informed.

Now we are past the answering the big questions part of the project, this part is the lets execute, so we can project manage that. There are many RFCs to update etc. its not all sets of instructions to follow, the Engineers will need to be super smart in their approach to completing tasks, but should have much better visibility to allow them to do that. No more isolation coding in teams never mind across teams.

A goldfish bowl, yes in many ways, who would subject themselves to that? Brave, honest and capable people I would think. Lets see how it rolls out though.

On the point of what we see, then we will use 2 systems, in one we will have a rolled up gannt chart that shows all subprojects and critical path, that will be point click and roll down to further detail about estimates, resources applied and so on, so you can drill down to great detail and with such detail you will see many changes week to week as we have things like delays, unplanned holidays etc. etc. In any case its going to be a lot to read and see.

44 Likes
18

Another great update Maidsafe devs.

Really love it how your never afraid to always grab the best tools (Rust/Quic/???) to get the job done.

@riddim and @dask :clap::clap:

Keep up the good work :vulcan_salute:

:stuck_out_tongue:

13 Likes
19

That is a lot more than what I expected. I promise not to be a whinger but a solid supporter even with the expected slippages. As the sub projects are ticked off, the excitement will no doubt build

9 Likes
20

Could the maidsafe twitter account tweet a link to this update? :slight_smile:

Bitbot is waiting to retweet SAFE Network Dev Update. :wink:

8 Likes
21

From point 1 perhaps is impt to know the real limits ā€¦ and option to have one instance that is known to be vulnerable just in order to seek where robust lies. The more real world evidence of ability to resist xyz will only help provide confidence. If thereā€™s and instance that tempts hackers to do their worst, perhaps that could be useful too?.. long-standing frustration of those attempts then helping the idea that the principal is even bigger and more robust.

2 Likes