MaidSafe Dev Update - December 21, 2017

Today, we are posting the last dev update of 2017. The team will be working intermittently over the next couple of weeks due to the holidays :christmas_tree: The next dev update will be on January 11, 2018.

Here are some of the main things to highlight this week:

Community Engagement Program

As we have been mentioning recently, marketing has been getting much more focus of late. A big part of our strategy moving forward is to engage much more with you guys – the SAFE Network community – to make the most of your skills, passion, and hard work for all our benefit. The Community Engagement Program, which we are launching today, is the first part of this process.

We have created a short video (with the help of @whiteoutmashups) and an infographic to help you understand how to participate in the CEP program and we hope to see many community members getting involved in the near future.

Initially we will be running one project only to help refine the process, we plan to increase the number of concurrent projects in the near future. The first CEP we ran back in 2016 produced the SAFE Browser and we are now inviting the community to submit proposals for a Safecoin animated video. We hope that this will raise the profile of the SAFE Network, it’s unique features and use cases once live. This should be a high-level video that acts as an introduction for newcomers who have little or no previous knowledge of the SAFE Network. The full Request for Proposal can be found here and we welcome questions and participations in the thread. Once you and/or your team have an idea, please submit your proposal in the Proposals category within the next 2 weeks. We look forward to seeing what you come up with!

Recruitment

As the development and marketing plans continue to roll out we are also placing a lot of emphasis on growing the team. With the impending opening of the Indian office in late January, we are looking to fill 7 roles initially to increase the capacity and range of skills within the front-end team. We are currently looking for two Java developers, two .NET developers, two web/UI developers and one designer with the Java and .NET developers also working on mobile.

Closer to home in the Ayr office we continue to look for a UI/UX designer, two Rust Engineers (preferably local), a QA/Release Manager, Customer Support Manager and another administrator. Some of these roles have proven to be difficult to fill, but we have involved a number of recruitment agencies to try and speed things up while still maintaining the quality of candidates.

We are starting to see progress on some of the roles so we are optimistic we will see some movement soon. It would be easy to rush and take candidates that we have doubts about, however, we have found in the past that this can cost more in the long run so are pushing the recruiters hard to get the best people possible.

SAFE Authenticator & API

A new patch version of the SAFE Browser (v0.8.1) is being released today which incorporates the following fixes (note the bugs were actually fixed in the safe-app plugin codebase):

  • Fix the safeMutableDataEntries.forEach function which was incorrectly returning the ‘key’ as an object
  • Allow MutableData handles to be removed from the safe-app plugin’s Map through a ‘free’ function
  • Minor fix in the DOM API documentation example for the safeCryptoSignKeyPair.getSecSignKey function

A new version of the @maidsafe/safe-node-app npm package (v0.6.0) has been published which incorporates a couple of minor fixes and it upgrades its safe_app library dependency to the latest version (v0.5.0). We have also made a minor reorganisation in the tests as well as increasing the test coverage to 90%+.

As a consequence of this, new patch versions of both the Web Hosting Manager (v0.4.4) and the email application (v0.4.3) are being released, mainly to upgrade their dependency with safe-node-app to v0.6.0.

The new version of the Web Hosting Manager also incorporates a fix for an issue recently reported by the community (thanks @bzee!) in relation to a convention for service names (more details can be found in this discussion on the forum).

This is the very first time we are able to generate 100% of the released packages automatically from our CI processes, which has not only saved us considerable time and energy but also gave us a good knowledge base to be able to automate such processes for other artifacts in the future.

Peruse has been updated with a number of enhancements and increased test coverage, as well as a larger behind the scenes refactor of the Redux store to enable sync across many window instances. This paved the way for History and Bookmarks pages in the browser, which are now enabled.

This is all available in the latest early-stage release, which you can find here, alongside a more detailed list of changes. This version is very close to being functionally equal to our Beaker Browser fork. We now need to make sure it is equally stable, as well as start putting some focus and effort on the UI/UX.

Beyond that, now that we have History/Bookmarks in place, we’ve been refactoring the save/retrieve state implementation for the new Peruse browser.

The JNI bindings have been integrated with the Java API. However, we are expanding the test cases right now and then we’ll update the documentation for the same. If anyone wants to try it out in the meantime, the JAR files for desktop can be built by following the README. The Android AAR files are not yet ready. The tests are not in place to assure that the Java APIs are stable. Considerable progress has been made this past week so expect tests and documentation to be updated soon.

Once the Java documentation is updated, we will be getting the C# API updated to be compatible with the latest native libs and have the mobile applications using these APIs also updated and published for Android and iOS.

SAFE Client Libs

We are releasing new versions of SAFE Client Libs: safe_app 0.5.0, safe_authenticator 0.5.0, ffi_utils 0.4.0, and safe_core 0.28.0. This release includes all of the improvements and bug fixes that were made to SAFE Client Libs over the last 3 months. Major changes in this release include: new cryptography functions in SAFE App, multiple improvements in the test suite (that helped us to catch and fix many obscure bugs), support for config files in SAFE Core (allowing to use in-memory mock storage and unlimited mock mutations), improved documentation (which is available on docs.rs), and many refactorings of the FFI functions. Overall, with this release, SAFE Client Libs has become much more stable and mature.

This release also includes some of the API-breaking changes that were mentioned in the past updates, but these affect only the Rust API and not the JavaScript or the browser APIs.

We are also continuing to work closely with the front-end and mobile teams to wrap up the auto-generated Java bindings. This week we have discovered some minor bugs (mostly related to naming conventions) that should be fixed very soon, so there should be more updates on this next week.

Routing & Crust

The last year has been full of hard work, but it is finally starting to visibly pay off.

Throughout this year, we have been focused on the design work. We were exploring consensus algorithms like PBFT or Tangaroa, and constructing our own, which resulted in the creation of the Ewok simulator. We had multiple brainstorming sessions and lengthy discussions, which were the source of most of our current ideas and gave us a lot of insight. All this work made us learn a lot about the minute details of how the network will function and what we want it to do. We were able to get a closer look at multiple designs and this allowed us to choose the one we think will work best.

Going back to present events - this last week was again full of thinking about the approaches to merging. After fleshing out all of the details we were finally able to come up with a proposal that we think will work. We also prepared a similar approach to splits. The full description is available in this Dev Forum post, but here is a short one:

Both merges and splits will begin with reaching consensus about the fact that such an event needs to happen. Next, in the case of merges, we need to additionally reach consensus about some details of the sibling section. Afterwards we start voting on blocks that will gradually change our current set of elders into the target one. During this part, since we are dealing with two sections most of the time, all peers that were elders or are supposed to be elders will vote - this way we can ensure that the relevant blocks will get a quorum of votes regardless of the order in which they accumulate (which can be different for different peers). Once we reach the set of elders we initially calculated as the target, we consider the merge/split complete.

In Crust, we have successfully integrated uTP into the dev branch. We have specced out our test plan and feature roadmap which will enable us to validate multi-protocol, randomised-port networking in various real life scenarios.

We had to make some changes to p2p as it was using a singleton object for NAT traversal configuration. Though nice for user friendliness it soon became an issue for testing. Rendezvous server integration into Crust was also pending. This week both issues were fixed: hard-coded contacts in Crust config are used as rendezvous servers and p2p uses a special structure (P2p) for NAT traversal information. uTP was already merged and direct connections seemed to work properly. Now that we are using the rendezvous servers we have a few ways to test peer-to-peer connections. We did however realize that it was difficult to integrate p2p’s UDP rendezvous server into Crust - uTP crate changes were required. These were promptly carried out and Crust now has the ability to rendezvous over UDP - an essential part of NAT traversal using UDP and subsequently feeding those hole-punched sockets to uTP for permanent connection-oriented communication.

We will also soon start with Crust integration into Routing. Routing master will be branched off and since Crust, in the new dev branch, exposes a compatibility layer (because there’s no immediate plan on refactoring Routing yet to work with Tokio event loops) the integration should hopefully not be hard - only the errors and some other types have potentially changed. Once this is done, we will spin up a few droplets and test for any regressions since we pretty much know what to expect from Routing’s master - Alpha-2 is running on it after all.

104 Likes

yay first!! 20 characters

10 Likes

CEP, well SOLID integration study would be a nice one, wonder who would be so brave @happybeing any ideas ?

35 Likes

Amazing accomplishments @maidsafe. Moving like a freight train now! I have so many questions.

21 Likes

Thanks Maidsafe devs (SUPER ANTS) for a GREAT year.
Thanks for all the incredible work you did this year to bring us all closer to SAFE.

Thanks to this 1 of a kind community, you girls/guys are really special that’s why I feel more @ home here than any other place on the current internet.

Keep rocking, keep your Maidsafecoins and yeah this year I failed to onboard 100K people to this forum, but I’ll have my revenge :stuck_out_tongue:

Praise the lord for the CEP (we’re finally back at it again, go super ants)

26 Likes

:hugs: amazing how you help to build the great things @whiteoutmashups

awesome! you said you’d want to grow the team - that really is going forward in large steps! =)

PS:

the randomized port part in combination with this completely decentralized structure still is something that is completely crazy in my head :grin::rofl::thinking::roll_eyes::upside_down_face::hugs: i love it!

13 Likes

yes, they do a Massive Amount of Insane Development.

Thank you @maidsafe for all the hard work done all year long ! I hope you will enjoy well deserved vacations with friends and family :slight_smile:

Indeed we could learn so much from a solid / safe junction experiment . Looks like a perfect candidate for a community driven project.

25 Likes

David and Team have been amazing in 2017… I am looking forward to 2018, the birth of The Safe Network!

Thank you so much for all your efforts :+1:t2:
Merry X-Mas :christmas_tree: and a Happy and Healthy New Year to you all ! :sparkler::fireworks::city_sunrise:

19 Likes

Boo. That video’s not working for me. Anyone else?

1 Like

Thank you all for your dedication and hard work! Merry Christmas and a Happy New Year.

9 Likes

Great work this year. Head to the Spanish Islands and get really drunk! Merry Christmas SAFE team!

9 Likes

Great year MaidSafe Team! Bring on 2018!

11 Likes

Major props to the team and community for the patience and sheer grit over 2017! Excited to watch and be part of progress in 2018 and beyond! Happy HODLdays :slight_smile:

12 Likes

Ok for me, are you still having issues? Here is the youtube link https://www.youtube.com/watch?v=LMhzv8oo2MA again, just in case.

6 Likes

Longtime lurker but I just wanted to say Congratulations to the team—you guys and gals are a genuine inspiration, inside of crypto and out.

12 Likes

Happy holidays to the MaidSafe team, and the whole community. A great year, and some amazing progress. I think 2018 is going to be incredible for MaidSafe (and the rest of us by extension). This project gives me hope for the future.

12 Likes

Glad it was half-way good enough to be used :slight_smile:

Future ones will be less rushed. Excited to work with you guys more in the future.

Let’s go 2018 :smiley: will be a great year

14 Likes

First post! Keep up the good work :slight_smile:

12 Likes

Welcome @tuner! Thanks so much to the Maidsafe team for all of your hard work this year! Now go have some fun!

10 Likes

Congratulations on a wonderful 2017

9 Likes