SAFE Network Dev Update - March 14, 2019

[maidsafe]

Summary

Here are some of the main things to highlight this week:

• We released a video with @JimCollinson as part of a new series of interviews with team members.
• We are starting the discussion on the RFC for RDF support in SAFE Client Libs, which is now raised as a proposal.
• @marcin made a post on the Dev Forum called Client Libs: Help Wanted! which describes the efforts of the SAFE Client Libs team to make it easier to contribute to the project.
• There is a SAFE Network meetup in Brighton tonight, @dugcampbell, @Pierrechevalier83 will be in attendance and @bart will be joining via Hangout.

Marketing

With a view to improving the DevHub, @dgeddes has been speaking with community developers, MaidSafe developers and external developers that are unfamiliar with the project. The objective is to identify common themes and then look to enhance the features and user experience for visitors to the site so they get what they need with the minimum of effort.

In amongst the ongoing knowledge transfer work, we’ve also today released the first video in a new series of quick interviews with team members discussing their background, how they came to MaidSafe and their daily routine. The first (willing) victim was @JimCollinson. They’re meant to be short and sweet so please let us know if you find them interesting!

@dugcampbell and @Pierrechevalier83 are heading to the SAFE Network meetup in Brighton tonight, where Dug will be providing an Introduction to the Network and @bart will be joining via Hangout to talk about PARSEC in its new fully asynchronous guise. It promises to be a great event; if you’re in town check it out.

SAFE API & Apps

Last week we added a few more tests to demonstrate how developers can use the Java API to perform client actions that will allow them to use features such as custom MockVault path, unlimited mutations etc. for a mock network. With this, we are wrapping up active development in the safe_app_java project until the next milestone.

We have been receiving positive feedback from last week’s release of the SafeAuthenticator mobile application with enhancements to the UI and UX. If you haven’t tried it out, head over to this forum post and give it a shot. Additionally, we have a bug reported regarding support on lower platforms. We have tackled this and a patch release is coming your way soon.

We are preparing a new release of the safe-app-nodejs package. It won’t bring anything particularly important but there are a few minor enhancements we’ve worked on since the last release that we want to make available. We expect this to happen next week.

A first safe_authenticator CLI proof of concept was implemented and we are now planning to have some internal reviews to better understand how to proceed. We are expecting people from the safe_client_libs team to helps us in reviewing it as it makes use of the Rust API directly. There are still many open questions in relation to how such a CLI will work with the rest of the front-end components and this is why we are still not sharing too many details on our thoughts, but just a heads up for those interested in such a thing.

The browser is receiving an Electron version bump after the Chrome bug exposed last week. This should make it into the next release. Alongside this, we’ve started to overhaul some of the tab logic to stabilise it and hopefully remove some of the niggling bugs therein. We’re also taking a look at how we’re incorporating the Rust libs into the repos to hopefully better support our CI systems. And we’re trying out a new end-to-end test system that may prove to be less brittle than our current Spectron implementation.

SAFE Client Libs

Today we are starting the discussion on the RFC for RDF support in SAFE Client Libs, which is now raised as a proposal. We are aiming to collect some feedback for it, iterate on the proposal, and after it’s finalised and accepted, we will start planning out the implementation.

We’ve also been expanding the wiki and added several new pages, including a high-level introduction to Client Libs. We hope the new introduction page, in particular, will make it more clear how Client Libs works at a high-level, something we realise has been lacking in our documentation until now. In addition, @marcin made a post yesterday on the Dev Forum called Client Libs: Help Wanted! This post describes our efforts to make it easier to contribute to the project and gives concrete steps on taking the first plunge into the codebase.

In continuation with last week’s work on RDF proof of concepts, @nbaksalyar has completely immersed himself with the integration of the SPARQL query language into an experimental version of SAFE Client Libs. We’ve made some good progress in interfacing with the underlying C libraries (librdf and Rasqal), but there are still some stumbling blocks concerning the thread safety. We already know how to resolve them though, so we’ll be continuing to work on examples which should demonstrate how developers can query, drill down, and get insights on the Linked Data stored on the SAFE Network.

Routing

This week the Fleming team took the focus more onto node ageing for their discussions. One decision reached was that we’ll associate work units with nodes and use this as a measure of when a node should be relocated. This is less random than the previous approach and shouldn’t cost any real overhead. For now a work unit will likely just be defined as handling a churn event, but this will give us scope to extend the definition later.

There were also discussions around new nodes joining the network; i.e. finding a balance between moving these less trusted nodes frequently and early, and on the other hand not causing the Network undue stress, especially when the Network is young. The team also agreed that we’d continue with the paradigm of not allowing more than a single node at a time to be relocated in or out of a section, since this avoids needing a section to resource proof multiple nodes concurrently.

On the PARSEC front, the reworking of the whitepaper has been largely completed, but we can’t quite share it publicly yet as we’ve secured academic review and are considering creating a formal publication. More details to follow in the coming weeks.

Regarding the code, a few further significant improvements have been made in performance. Possibly the most significant, as mentioned in last week’s update, was the idea to bundle multiple observations into single gossip events. This was done in PR #273 and yielded more huge improvements in some of the benchmarks. Other than this one, @Jean-Philippe has done some excellent work in the last few weeks profiling the code and making the majority of the recent PRs which improve performance.

Crust

As we continue evaluating the requirements from Crust for Fleming, in parallel we’re looking into detailing out the tasks for dynamic connections and looking into the quinn crate for figuring out missing components and how those can help us. Dynamic connections, among other things, presents the illusion to the upper layers that Crust is connected to 1000s of peers when in reality Crust is actually doing quite a bit of work to make sure we don’t run out of resources by constantly shuttling peers between an active and a passive list. The QUIC protocol, on the other hand, has built-in features like multistreaming and has an ability to use a single UDP socket for as many connections as we want thus keeping the socket-descriptor usage to a bare minimum. There are some pending issues in Quinn like here and we are monitoring if it has enough of what we need. We are setting up a proof of concept to test this and if it’s stable and suitable for our purposes we would be in a much better position to decide whether to go for our own implementation of dynamic connections or switch to QUIC instead in the road to Fleming.

[anon78865233]

Brilliant update! Now onto reading it!

[andyypants]

second post of the thread!

wondering how market will react to this one. I think there’s been some buying in hopes that we would hear PARSEC was actually ready to deploy for testing. Could be a bit of backtracking now given this sounds more like conceptual stage stuff. That meetup will hopefully kindle the flames, but prolly only among people that are already interested anyways. These videos are nice primers to get some new people interested. I think that the marketing team really needs to put some focus on bringing in new people, or reminding people that were interested like 3 years ago that a truly decentralized network is around the corner. We can all play a part in that though, so tell your friends!

[anon94252342]

nice update, but cant find anything about whats now the focus for the Parsec team ? Isnt it time for testing, test nets?

[upstate]

@maidsafe

since this avoids needing a section to resource proof multiple nodes concurrently.

Did this change? I thought vaults were going to be waiting in the wings to be add when more were needed?

[nicklambert]

There is no Parsec team as such, more a Fleming team and a Routing team. Moving forward they will be continuing to discuss node ageing for Fleming, engaging with academia on the paper and discussing strategies for publication, Malice Detection for Fleming and documentation.

[anon94252342]

Oh ok, so no testing yet in sight? Read about getting close to test nets already like 2 weeks ago in the update, but ok so maybe we are further than we were then…

[nicklambert]

We’re not further away than we were 2 weeks ago progress on Parsec performance has been good, focus is on Dynamic Connections and Node Ageing as the big ticket items in relation to Fleming

[Bogard]

Not sure about the approach here. Unless the academics in questions are advising, if you want to publish, you’d just find a suitable journal and submit the paper to them. That journal then takes care of finding peer reviewers (typically at academic institutions).

Still, publication is moving from gatekeepers (journals) more to preprints. Depending on what journal it is, you probably don’t want the parsec paper behind a paywall. So a preprint where any expert is free to comment and offer suggestion may also be an option. Of course let’s not forget that the bitcoin whitepaper never needed a gatekeeper’s seal of approval. Does parsec need one here? (I know there are potential benefits, but are they outweighed by the potential negatives)

[tobbetj]

I think you might confuse the statement alittle, from what I understand PARSEC works and has been tested. The academic paper is for another purpose than which is needed for the network to work. From what I understand with the latest findings in making PARSEC fully asynchronsus (which is not needed for the network to work), it brings a final piece to complete a academic paper.

[andyypants]

oh ok:) wish it was more clear here. Hate to see the momentum in price fizzle. I think its people anticipating a real decentralized network soon as they integrate PARSEC and can finally trust more then just highly vetted peers. Maybe I am totally wrong on this but it sure would not hurt if they wanted to hint this was coming soon!

[nicklambert]

@Bogard

you probably don’t want the parsec paper behind a paywall.

Agree 100%. We’re looking at different open access models.

@Bogard

Not sure about the approach here.

As we mentioned in the OP we’ll update more on this soon, things have been pretty fluid this week. Regards peer review, the more eyes on the paper the better and it will add more credibility in certain circles which will do the technology no harm. I don’t see too many down sides tbh.

[Haigha]

@maidsafe

In amongst the ongoing knowledge transfer work, we’ve also today released the first video in a new series of quick interviews with team members discussing their background, how they came to MaidSafe and their daily routine. The first (willing) victim was @JimCollinson. They’re meant to be short and sweet so please let us know if you find them interesting!

Interesting in how unprofessional it is. I am by no means a professional but just wow. I actually wanted to do this same idea via sound a couple of years back. I spent quite some time preparing a proposal and shared internally with Maidsafe. There were concerns about my experience and if it would come out professionally, which was disappointing but I guessed justifiable. Now near two years later this is put out. Looks like this took all of ten minutes to prep and make. Some issues:

1. Video is off centre
2. Someone is talking in the background.
3. Someone is feeding cue cards on our right and seems to be being funny while doing it.
4. The amount of head bobbing, “mhmm”s and “yeah”s is pretty distracting.

No offense I hope just my 2 MAID.

[JimCollinson]

I know! Normally I’d get at least half an hour in makeup.

Seriously though, this was just a wee chat, in the middle of our work day in the office (hence the noise), no scripts, no glamour. Just to let you guys join us for a bit and get to know everyone and what they do; face to a name and all that.

I’m sure there will be more interesting ones to come

[dirvine]

I think it works very well and shows honest hard working no-BS people doing what they really are motivated to do, not for wages alone, but because it is right. I could not say it better than oor Rabbie

“an honest man’s abin them a’”

It’s hard to be brutally honest and do the right thing, it’s easy to critique, I am glad we choose the hard way

[bochaco]

@Haigha

Interesting in how unprofessional it is

Do you really mean unprofessional, or you are talking about the informality of it? many people tend to believe that they are the same, in fact I believe we need to get rid of unnecessary formalities, specially in projects like this one, fomalities put many people off and even prevent them from sharing their most precious ideas and knowledge, informality also inspires people to be more honest I believe, simply because that’s how humans really interact with each other.

[david-beinn]

Well I thought the video was good, and interesting. Nice to see one where people look a bit more relaxed, just talking to each other. I think it’s a good model for future videos. The video dev update a few weeks ago was almost there, and with this extra bit more relaxed feel I could really see that working well too. I know the golden rule of the recording studio is to keep the mistakes, but I must admit the talking in the background was a little bit distracting at times. Other than that, thanks again to all, and keep up the good work!

[Bogard]

Totally understand. The main downside I see is that it may slow publication. While academia does add credibility, it typically moves at a glacial pace and can introduce all kinds of delays. There are other potential avenues to get credibility without suffering from academia’s slower pace, e.g., have them review the manuscript after preprint/open source publication. But I’m sure you’re considering all of these factors. It’s also good to engage them prior to putting it online to satisfy the big academic ego. Good luck with the process.

[Haigha]

@bochaco

Do you really mean unprofessional, or you are talking about the informality of it?

I mean unprofessional:

@Haigha

• Video is off centre
• Someone is talking in the background.
• Someone is feeding cue cards on our right and seems to be being funny while doing it.
• The amount of head bobbing, “mhmm”s and “yeah”s is pretty distracting.

Seems offense was taken on my so-called “smarmy, dishonest, opinionated” feedback. What the hell? I am glad I took the hard brutally honest way as it seems to have shown some true colors. If so easily offended don’t ask for feedback and then shit on me because I gave negative honest feedback.

[Nigel]

@Haigha

What the hell? I am glad I took the hard brutally honest way as it seems to have shown some true colors. If so easily offended don’t ask for feedback and then shit on me because I gave negative honest feedback.

I think people took it as retaliatory because Maidsafe hadn’t taken you up on the offer awhile back when maybe they just recently decided within marketing that it was a good time to start producing these. I think your feedback is useful just maybe not the delivery.