SAFE Network Dev Update - September 13, 2018

Summary

Here are some of the main things to highlight this week:

Marketing

A few things to highlight on the Marketing front this week. First up, we’ve put up a new Forum post providing a little bit of context for each of the SAFE Network Fundamentals that were first shared a couple of weeks ago. Hopefully this will start to flesh out a few of these concepts, particularly for newcomers. We see each of these as crucial elements that we have used to guide the design and implementation of the Network and we’ll get these added to safenetwork.tech within the next fortnight.

As you might have seen earlier, we’re delighted to announce that ChainRift will commence MAID/BTC trading pairs from Monday 17th September at 18:00 GMT. We’ve set up a new topic to collect any questions that you might have and the team from ChainRift are keen to answer any questions there. Head along to the post announcing the listing if you’d like to learn more.

This week, @dgeddes also relaunched the RFC process. For those that aren’t aware, the Request For Comments process is a recognised way of encouraging others to get involved in making suggestions around the technical implementation of specific areas. As with every open source project, we want the best ideas out there to be adopted so if you can help as we move forwards, please do check out the post to learn more.

A quick update on our MozFest application. Unfortunately, we’ve just found out that our application to run a session on decentralising the web and setting up a SAFE website didn’t make the cut for the event this year. A real shame as there’s a clear overlap in interests there - but no fear, we’ll apply again at the next opportunity.

Finally, in terms of talks this week - @dugcampbell opened up proceedings with an intro to the SAFE Network at a Manchester Blockchain Conference and also spoke at a workonblockchain.com event (you can check out the video here) where @pierrechevalier83 and @bart dug deep into PARSEC for an audience of interested developers. We’ve edited the main PARSEC video talk into sections so that we can make it as easy as possible to digest for those who want to go into depth. You can view the playlist here and see the slides here.

SAFE API & Apps

Yesterday we released a new version of our Peruse SAFE Browser (v0.7.0) and Web Hosting Manager (v0.5.0). As mentioned in that post, we are planning to deprecate our previous Beaker-based SAFE Browser early next week, and all websites will be updated to direct you to this new version as the only available SAFE Browser for downloading.
As anticipated last week, this new version of the SAFE Browser also includes some experimental APIs we’ve been working on recently, e.g. WebIDs & RDF utilities APIs. Even though this is not already implemented in this version, from the next version onward, the SAFE Browser will disable any experimental API/feature by default, and they will be optionally enabled when a special flag is provided when launching it. We are also adding more details in the README file itself about the new DOM APIs, as well as details about the experimental APIs & features we are exposing, so developers can better understand how to use the API and also be aware of what is experimental. This will be important as experimental APIs & features could be changed, deprecated, or even removed, without much notification in subsequent releases, and developers will need to be very aware of this when creating their applications. We will provide more information on how this will work exactly in the next release.

As always, we appreciate your feedback, and especially to get issues reported so we can work on fixing them. When reporting a bug, please do it directly in our GitHub repository since that’s where we are now handling all this and having detailed discussions about the code. If you otherwise have issues which are not necessarily bugs, or would like to get clarifications before being certain of it being a bug, please post your question(s) in our Dev Forum where people can provide guidance and help.

The past week safe_app_java has had a spring in its step as we have resolved the Android/JNI issue that has been holding us back. Now our work on the Android library is back on track and we look forward to the exciting weeks ahead.

The SafeAuthenticator mobile app has been moved to a new repository. A pull request is under review to change the project structure. Also, as we mentioned last week, we will be cleaning the safe_mobile repo (now renamed to safe-email-app-csharp) which will now only contain an example email app.

SAFE Client Libs

This week we completed a first draft of the integration of SAFE Crypto into SAFE Core. Doing so has made us aware of some missing functionality which SAFE Crypto currently lacks but is required by SAFE Core. We’ll be implementing the missing pieces in SAFE Crypto soon, including the ability to generate keys from seeds and to explicitly use nonces when we need control over the specific nonce that is used (e.g. the IV in our self-encryption algorithm).

We also worked closely together with the Mobile team to resolve the long-standing Java/JNI bug that manifested itself only on the Android platform. The native SAFE Client Libs functions were called but the callbacks never got returned. We extensively tested the code and finally found the root cause of the bug: it turned out to be the way class loaders work in the Android JNI environment. We didn’t account for the fact that when looking up Java classes by their names, Android JNI attempted to use a class loader from the last Java class available up in the call stack (e.g., if you call a JNI function from a class HelloWorld, then HelloWorld.getClassLoader() will be used). This didn’t work when we tried to call the Java callbacks from inside the Client Libs on a different thread of execution: there were no Java classes on the stack and thus the class lookup failed. We fixed this by caching a valid class loader and using it for our class lookups, and it worked like a charm. Now, the required changes in SAFE Bindgen and FFI utils creates are already merged and published and we should be publishing a new version of SAFE Client Libs with fixes soon.

The new version of Client Libs should also include some useful internal changes that don’t break the existing API. First, we’ll switch the IPC encoding scheme from Base64 to Base32. The primary reason for this change is the wider and easier cross-platform support (the front-end team discovered that they needed to have some workarounds to cater for case-sensitive Base64 encoding). We’re going to support both encoding schemes to maintain the backwards compatibility with existing apps based on older Client Libs versions, but we’ll be defaulting to Base32.

The second expected new feature is the more clear distinction between the mock and non-mock modes of operation. For example, an app using the mock network mode could request to be registered with the Authenticator connected to the real Alpha 2 network. Subsequently, the app failed silently in this case because of a disparity in the source of data, as the app attempted to use the local MockVault which contains no expected information. Thanks to @happybeing for reporting this case on the Dev Forum and suggesting to fix it by returning an error as early as possible. We have created a Jira task and we’ll include this in the next Client Libs release.

We have also started documenting some of the internals of SAFE Client Libs which will be useful to our end users. For example, this week we documented the GETs and mutations performed via our API, in response to @happybeing here. We believe that this information will be useful to app developers and we’ll likely integrate it into our docs.rs documentation of SAFE Client Libs.

Routing

Last week we’ve been focusing on dynamic membership. With Adam reinforcing us and Bart back from holidays, we’ve really picked up pace and taken a big bite out of it, merging MAID-3029/3032/3033 to handle Peer adding, and MAID-3013 to handle Peer removal. Full steam ahead! The main remaining task for dynamic membership to be called complete is to test the new code to be sure it behaves as expected.

Ongoing in parallel to the dynamic membership work, we’ve finished the work on parsing event graph data from dot files (MAID-2970) and constructing PARSEC instances directly from these (MAID-2974). These together allow us in turn to create a functional testing framework based on dot files (MAID-2975) which we think will greatly help us write functional test cases. Working and thinking in terms of visual graphs is almost always easier and this will mean we will be able to feed them directly into tests. Exciting!

While the testing framework gets built, we are now also starting to tackle tasks related to malice detection and handling, with MAID-3066 and MAID-3069 being first out of the gate. These involve making sure that the genesis group, the initial set of members in a section, is robustly handled in the presence of malicious nodes.

On the community front, @bart and @pierrechevalier83 presented a Deep Dive into PARSEC at a “Work on blockchain” meetup in London. The talk has been cut into sections and you can view the playlist here and see the slides here. We also had the pleasure to meet up with @dugcampbell and a few members of the community: @Charles-Johnson, @opacey and @sonder.

77 Likes

First fort the first time

14 Likes

Ahh, What? Today is Thursday, Sept. 13th, which Monday are you actually referring to?

6 Likes

Oops this was a typo, I just fixed it. It should say Monday 17th September.

14 Likes

Shame about Mozfest. Did they give any feedback?

12 Likes

Good to hear java progress! Routing too! Loved the fundamentals also! Great update!

14 Likes

Thanks for the great update! We appreciate all of the hard work.

8 Likes

Nothing beyond the GH comment unfortunately John.

12 Likes

This project continues to make great strides, amazing work! And I like the vision and partnership with ChainRift. It pays to be selective and partner with exchanges that are also passionate honest, and have a plan for a reliable future.

13 Likes

Always thanks for your update!!

6 Likes

Without code this project is not much (but an idea), so thank you Maidsafe devs for your hard work. @maidsafe maybe this can help the super ants with de :bug: ging :sweat_smile:.

Mozfest :crazy_face: bad :robot: GH

6 Likes

@dirvine is the safenet team working on alpha3 100% focused on parsec milestone2 right now or is there any work made in parallel covering the coding etc. for message relay and other routing stuff that is required for the alpha 3 to be live?

The latter, that should be clear in the updates though.

5 Likes

Thanks David :slight_smile: 20char…

in jira there are many unresolved tasks related to milestone2 , according to the update the main remaining task for dunamic menbership aka milestone2 is to test the code, but i see alot off open tasks it seems like the few dosens of tasks to be completed will take few more weeks at least… or am I wrong here

That’s because the scope for milestone 2 is larger than just dynamic membership, even though dynamic membership is the main aspect.

Roughly, the scope of milestone 2 includes:

  • dynamic membership
  • malice detection and PARSEC-side handling (some handling will be done in routing as part of the next milestone)
  • production ready testing
  • production ready documentation
  • production ready performance analysis and optimisation

The idea is to have PARSEC be “complete” as far as software can ever be complete.

Currently, we are on the hot path for dynamic membership (excluding all the other stuff) as we reckon there is value in integrating a non-malice-proof (as in: malice is not punished. PARSEC is still resilient to < 1/3 malicious nodes in a section by construction), non-“production ready” PARSEC in routing to start testing the happy path works before covering all the milestone 2 tasks.

In other words, expect to start seeing visible progress before milestone 2 is over :wink:

17 Likes

thanks a milion pierre :slight_smile:

4 Likes