Thanks to the entire team (again) for their contributions to the update this week. Pssst! the Routing example is functioning as expected… that statement seems a little underwhelming in itself, but trust me: serious time and effort has gone into getting to the stage of being able to say that again, I shall let the Devs in Routing bring you up to speed with the good news.
##Routing
As mentioned last week, after implementing the basic functionality documented in the new flowcharts, we took a further pass through the flow of messages with an eye on security and further reducing vagueness / complexity. Thanks to daily hangout sessions running from 06:30 this has now been completed with additional security validations included per message hop.
Running multiple instances of the key_value_store example confirms the expected behaviour with each node’s routing table getting populated and updated as they should. Running a further instance as a client shows that putting and getting is working as expected, albeit we’ve only run the small testnet on a single machine so far.
Having node discovery working i.e., routing table updates, removes the most problematic bug that was present from before the latest refactor. Full churn handling while not switched on in the implementation at the moment is not predicted to be a major cause for concern as churn and refresh have already been tested and proven prior to the work we just carried out. We anticipate that reinstating the logic will be relatively pain free. Brian is working on this now as the rest of the team have moved to confirm vaults are OK with the revised API. .
We’ve also slightly refactored the API (as mentioned earlier) to make life easier for Vaults and Core which interface with Routing. The API functions used to return local errors and network errors as asynchronous events, whereas only network errors really suited this paradigm. Take a Put request, for example. The client would call Routing::put and this function would return nothing. If a local error happened before the actual request could be sent to the network (e.g. the node isn’t connected to the network) this would be reported to the client in a separate error event. This is fine for a network error (e.g. once sending the data, the client’s managers reject the request because the account doesn’t have enough safecoin) since there can be a significant delay between making the request and finding out that it failed. But for local errors, we find out more or less immediately. So the API has been changed to have the Routing::put function (and all other functions) return any local errors right there - which saves the client from figuring out which particular request a subsequent error event related to.
Current work is now focused on documenting the updated source code, re-enabling churn and refresh and updating/adding tests. uTP integration remains to be done, tying in the recent changes to Crust. Also Core and Vault updates are necessary due to the changes introduced in the Routing API mentioned above. Overall the codebase is smaller and cleaner now and should simplify the process of feature inclusion. We hope to move to using the utp / NAT traversal API from Crust very soon all going well in Vaults (if we can confirm Vaults tomorrow). These are ferociously tight timescales we have internally. We hope this level of commitment will satisfy those who wish timescales to be better put on the forum etc. This current work is not project work in terms of the sprints, which are pretty much specced and allocated time as we move on. This is a rapid attack on some code that went sideways and therefore cannot be succinctly evaluated in terms of completion. None of us like this speed and pressure so the sooner it’s complete and we are back on track the better for us all. We percieve this will be very soon now and we will know even more in next update, if we are not finished by then (we can always hope, but it’s very doubtful for next week).
##Crust
Last week @qi_ma was diagnosing the service::test::network_test hanging in Windows. After some debugging into the cause for this issue, we realised this was coming because of issues in the rust-utp multiplexer thread from UtpWrapper. With the rust-utp module from upstream not supporting read and write concurrently, we’ve had to introduce this wrapper object to expose the same functionality to Crust. A quick fix to resolve the hanging issue was submitted via Crust PR 442, which breaks the infinite loop (causing the hanging) after a mandatory time out period has passed. However, reviewers spotted that the PR modifies expected behavior (you can catch the full discussion in the linked PR). To maintain the expected design, a suitable approach needs to get implemented here with proper notifier / correspondent messages. Also required is to keep sending a keep-alive or ping-pong style msg to detect connection status. With @vinipsmaker looking into the rust-utp issues, He’s going to be taking this PR and updating it to the approach best suited for Crust.
@vinipsmaker investigated the issues around uTP in Crust and rust-utp. we think we may have found the issue in rust-utp and it’s related to timeouts that are not reached (explained portions of the issue we’ve had with windows hanging tests too) and we are not resending packets because of that. Currently he’s working on implementing a suitable fix for the same which should let us know if this was the issue and hopefully have it addressed accordingly.
##Client
This week we continued with the work to standardise the way web applications interact with the SAFE Network. The initial restrictions imposed by the browsers have been identified. Out of the four browsers we are investigating (Firefox, Chrome, Safari and Internet Explorer); Safari has the most restricted addon environment, it does not provide any option to communicate with the Launcher through an add-on.
So alternate methods to overcome these restrictions have been investigated and documented and these alternative approaches and the add-on strategy taken by other similar applications are currently being analysed. One more challenge here is to try ensure that our agreed approach will work with all platforms from mobile to desktop, ensuring that the app devs don’t have to cater for each platform separately.
Launcher on mobile platforms has not been specified yet, further work is required to establish whether a similar approach would also be a good fit. All of these are ideal behaviours to have, but as we explore more we discover that limitations across varying platforms never fail to surprise at every stage of the process. It is essential now to complete the exploration, investigation and standardisation and lessen the chances of hitting surprises further down the line that then hinder the progress at that later stage.
##MaidSafe Website
After the completion of design and implementation of the end user site, it is now being reviewed internally, with the help from a couple of community members. I would anticipate the site going live toward the end of this week, at which point we will optimise the site for search engines as well as setup and utilise some of the tools that exist with Google analytics, to allow us to improve the user’s experience on the site. Once live, we look forward to making iterative improvements and if there is anything you would like to see on the site, please let us know via the forum.
##Documentation
Work has been completed documenting the SAFE Launcher, specifically using the CLI of Launcher and SAFE NFS and SAFE DNS examples and this is currently under internal review. The API documentation is also being worked on with code examples still to be completed here, this should be finished and shared with the community very soon.
Overall this past week has been challenging, but very worthwhile and we are pleased to have some very positive outcomes to share. With Routing now working against the example, we will look to enable churn before turning our attention from tomorrow to Vaults, which will need to be updated to correspond to the changes made in Routing. This week we also plan to get the droplet testing going with Client and in the Crust library we shall continue to focus on integrating uTP; with all of these in place and working as intended we will be able to present you with the Rust 5 deliverables. As David suggested on the forum, this is weeks away rather than months and as soon as we have reviewed Vaults and have a handle on the
uTP integration we will be able to tell you more. Thanks again for all the continued support; Rome wasn’t built in a day, and when you are making something that has the potential to be truly world changing, it does take a little time.
