MaidSafe Dev Update - 7th June 2016

Guided by the team leads, the devs continue to focus on the tasks that need to be completed to move the SAFE Network forward. As always, thank you to the team leads for taking the time to provide these updates - so without further ado, on with the update…

##Crust, Core: Andrew, Spandan (tl) & Vinicius.

Crust now functions correctly with direct connections. The team have completed the integration with Routing and ran tests which create over 50 Routing nodes on a local machine and everything went smoothly. We tried contrasting this async with the previous blocking version of Crust and observed that (for starters) the async version has exactly six threads per Routing node compared to almost ten times more in the non-async code. These threads directly translate to CPU load, with a greater number of threads equating to higher CPU load. The blocking Crust has the CPU screaming with 1000’s of threads in total (for 50 nodes) while the async version is very benign with almost 10x less - we only occasionally hear the fan kick in indicating CPU is relaxing :wink: . Although it would be a little hasty to mention anything about benchmarks at this early stage it is interesting to note that async ran a little over a minute faster than the blocking Crust.

Hole punch tests are all passing on Linux and OSX, but are causing random issues on Windows. With a few team members off on leave last week and absent for the beginning of this week, progress has slowed slightly here. We are currently working to resolve the hole punch issues on Windows (mainly due to crashes from inside the mio crate), and once this has been completed, the team’s focus will shift onto prioritizing message sending globally across all peers so that Routing gets an API through which it can specify which messages are most important. It should also preempt the transmission of lower priority messages which will either then be discarded, or transmitted sometime in future, if it is still valid to do so.

A while back we found a bug in the mio code. The crash site is due to an unwrap() in the Windows part of the code. While miow (mio uses this internally for windows) returned an Option indicating there is a possibility of None, mio simply unwrapped this leading to the panic. With that sorted out (in our fork of mio currently) we are progressing further with hole punch debugging on Windows. Once completed we will inform the mio maintainers about this.

##Routing, Vaults: Adam, Andreas (tl), Fraser & Qi.

We are still mostly focused on cleanups and Crust, however, the guys have now also started to port Routing to the current Crust “async” branch to begin testing and we are seeing promising results so far! We have added a new message type so that clients can retrieve their data chunk count and allowance.

Next we are going to attack some efficiency problems: Routing messages of up to 1 MB in size use a lot of bandwidth in the network and create a lot of lag, so we will take two measures to ameliorate that problem:

  • Large messages will be split up: Instead of waiting for a whole megabyte to arrive before starting to send it on to the next hop in a message’s route, for example, nodes will be able to start transferring the first small chunk(part) of it on to the next hop immediately, while downloading the second one. Also, important messages will be able to squeeze in between the chunks instead of having to wait for the whole file transfer.
  • If a group sends a large message to prove to the recipient that there is a consensus on that message, only one member will send the message itself and the others will just send a signature.

##Client : Krishna (tl), Scott & Shankar.

The website was updated and is now live with few style enhancements. For those looking to brand up their social media profiles, the new brand assets page may be of interest.

The RFC for launcher version 0.5 is being discussed. @cretz has been very helpful with suggestions here. If there are no more changes / suggestions in the next couple of days, the team will begin planning the implementation and update the status from proposed to active.

The major points covered in the RFC are:

  1. Removing encryption between launcher and applications.
  2. Improving security validation from launcher and proxy.
  3. Refactor NFS API and also include streaming abilities.
  4. Presenting the user account storage stats in the launcher.
  5. Configuration for enabling and disabling the proxy from launcher even from login the page.

We also plan on improving the UX of launcher based on the issues that have been reported and also from our own internal list of bugs and desired enhancements.

The CI integration has exposed few bugs in the APIs. We will fix these issues and complete the CI integrated for SAFE launcher. The planning for the RFC will be completed this week, with implementation work starting soon after so the client team expect the next version of launcher to roll out pretty soon.


In summary, with the Crust mio change and the efficiency enhancements in Routing, we will be starting vault droplet tests tomorrow. All things going well this will lead directly into community TEST 4. So great progress this week, but we do need to verify that we haven’t had any code regression.

Thanks as always for all of your support, here is a link to the weekly transcript.

55 Likes

those logos at the bottom of the new page really are intuitive and i like them a lot :slight_smile:

5 Likes

Thank You for the update!

4 Likes

This is awesome!!! :grin:

10 Likes

This is great news. If devs start to build on top of Crust/SAFE you want the low level stuff to run smoothly in the background. David made this part about the hole punching and stuff very clear in the interview. All great news that it’s coming to an end. Gotta love all these routers and NAT’s :nerd:. Great thumbs up to the team :thumbsup:.

Question: this splitting up messages is useful for messages but not for chunks? I remember a disperse algorithm?

12 Likes

This is incredible! AWESOME!!

5 Likes

WOHOOOW Testnet4 Coming!!! :smiley:

9 Likes

The ants metaphor has never been clearer!! :laughing:

5 Likes

Thanks so much for the update. Keep up the good work!

5 Likes

I wish I read slower… :slight_smile:

Much appreciation to all! Peace!

5 Likes

awesome, i’m really diggin’ that asset page. looks neat with those wallpapers!

6 Likes

I just ordered a new custom flexfit cap the other day with the safecoin logo embroidered on front…turned out great!!!
Can’t wait to see how these all get used! :slight_smile:

4 Likes

Thanks for the update Ross and Maidsafe devs all I read is progress.

Hopefully this week we can start with test4 :stuck_out_tongue:

Keep up the good work

5 Likes

I love the downloadable images and wallpapers! Sweet! Totally using these

5 Likes

Sounds like they took some ideas from Bitcoin xt thin, I believe? They do the same thing, and it progress the hops at a much faster rate.

1 Like

I’m glad someone is looking into mio Windows bugs. I have reported couple of them as mioco was suffering crashes on CI Windows builds, but couldn’t fix them, as I’m not a Windows user.

2 Likes

Is that 50 instances of safe_vault running on one computer? That’s pretty amazing if it is. It opens up interesting possibilities, such as easily simulating a large SAFE network on a small LAN ( @tfa ), and experimenting with chunk obfuscation scenarios ( @cretz @dirvine ), each in its own Docker container for a realistic isolation.

12 Likes

yes …
[Edit - correction 50 instances of routing object I think, vaults do not add that much though]

8 Likes

Would it be reasonable to make this a configurable option defaulting to ‘off’ rather than removing it altogether?

4 Likes