Here are some of the main things to highlight this week:
- We originally planned to make all our core libraries (such as Routing, Self Encryption, etc.) GPLv3 only. However, we have now decided to make some exceptions: for instance, our connections networking library Crust will be available under MIT and BSD licences. Many other utility libraries are also now going to be made available under this dual license.
- Ravinder (@ravinderjangra) has joined us at the Chennai Office. He has good experience with C# and he is an active contributor to Xamarin.Forms. We are looking forward to working with him.
- The bindgen branch has been merged upstream. This hits a big milestone in regard to the language bindings support.
Marketing
As many will be aware, we are developing a SAFE Academy website as a resource to help people learn more about the technology behind the SAFE Network. The first version will initially be a stripped-back version, with the rest of the content being built out over time. After working on this project intermittently over the past few months, we now have an internal review of the first version of the SAFE Academy scheduled for the end of next week. All being well, we should then be clearer on a projected launch date for the Academy.
As mentioned at DevCon, we have amended our licensing strategy to remove our commercial licence and apply an MIT and BSD license to our APIs and bindings. As some of you will have seen, during this last week, Nikita and other members of the team (Marcin and Srini) have been making these changes in GitHub. We originally planned to make all our core libraries (such as Routing, Self Encryption…etc…) GPLv3 only. However, we have now decided to make some exceptions: for instance, our connections networking library Crust will be available under MIT and BSD licences. We have decided to do this for this library because it’s a highly useful library that will help a number of other decentralised projects. By applying permissive open source licenses, it will enable developers who fork Crust to have complete flexibility in licensing their derivative work as they see fit. This should in turn lead to more developers using Crust. We believe this move strikes the right balance between ensuring that we lock the fundamental components of the network open for all, whilst also giving some highly useful tools back to the Rust and decentralised communities. We are very much looking forward to seeing what they create with it. To support these efforts, we will start to make more developers aware of Crust’s features and benefits in the coming weeks which can only be good for the SAFE Network and the community in general.
In addition to Crust, we are also switching the following utility libraries to the dual MIT/BSD license: config_file_handler, fake_clock, maidsafe_utilities, lru_time_cache, accumulator, resource_proof, get_if_addrs, and secure_serialisation.
SAFE Browser, Authenticator & API
As part of our tasks of making necessary changes to upgrade the safe_app
library in our safe_app_nodejs
package, we were working on some changes that affect the MutableData entries API. The forEach
function exposed to iterate through MD entries has been replaced by a new listEntries
function which simply returns the list of entries as an array of JSON objects. This was a pending change to have a coherent API as the other functions to iterate through MD keys and values were also removed in previous releases of the safe_app
lib and safe_app_nodejs
package.
We have additionally exposed a new function to simulate a network disconnection event which will allow developers to test the logic their apps are expected to follow upon such an event. Testing this type of logic will now be possible even when using mock routing by invoking this function.
Along with these changes in the API, we are reviewing and trying to make some minor changes to a couple of functions to align them to the terminology and names used by the underlying native library, as well as trying to make sure the safe_app_nodejs
API is coherent with the other language bindings we are working on. We will provide a full list and details of the changes in our changelog when we release this new version so developers can easily adjust their code if necessary.
One of the features we are now also working on is to allow an application to be able to set the metadata for a file created using the NFS emulation, i.e. the metadata that is stored in the NFS container (MD). This is already supported by the safe_app
library so we are just trying to expose the functionality in our API.
Ravinder (@ravinderjangra) has joined us at the Chennai Office. He has good experience with C# and he is an active contributor to Xamarin.Forms. We are looking forward to working with him.
The initial integration of the Java APIs with the bindgen code is going well. The test cases are being updated to use the Authenticator API for end-to-end testing. The next step is to test the Java API on all three (Windows, macOS, Linux) desktop platforms. Once the tests are confirmed to pass, the packaging of the JAR for catering the different combinations will be generated. We are also planning to work on a simple example application to test the usage of the packaged JAR files.
SAFE Client Libs
As we’ve changed the license for SAFE App from GPLv3 to BSD/MIT, we are continuing to work on providing a full API for app development contained in a single library. As was explained in the previous update, the primary reason is to ensure the completeness of the Rust SAFE App API: it should be the only dependency required to be included in the Cargo.toml
file to communicate with the SAFE Network from Rust apps. The pull request is raised and being reviewed and the progress can be tracked in this JIRA task.
The bindgen branch has been merged upstream. This hits a big milestone in regard to the language bindings support. And now that the front-end team confirmed that they haven’t found any major issues with the generated bindings, we are moving to the next step: automating the deployment for mobile platforms. This task will entail extending our continuous integration pipeline and including builds for iOS and macOS platforms in addition to the existing ones.
Routing
Following up from last week’s dev update, the Routing team is still working on tackling Eventual Consistency.
TRIVIAL
In a comment on last week’s dev update, @Antifragile linked a recent paper: “Byzantine Agreement, Made Trivial” from Silvio Micali, one of the authors of Algorand.
This paper describes a way to obtain Synchronous Binary Byzantine agreement without the need for a Common Coin. They replace it with a “Concrete Coin” which is a function that will behave like a Common Coin probabilistically often (~ 2/3 of the times).
They then split the decision process into rounds composed of three steps so that imperfect Concrete Coin is enough for reaching binary consensus.
The paper in itself presents, as advertised a rather trivial algorithm; partly because creating a Concrete Coin in a synchronous setting is indeed very easy, and partly because the three steps algorithm needed to converge with a weaker coin is also quite straightforward in a synchronous setting where we can simply wait to hear every node’s voice at each step.
For our asynchronous usecase, it is slightly less trivial, but we think it is possible and actually less difficult than getting a fully asynchronous Distributed Key Generation scheme to work for our decentralized use case. On a side note, we would like to thank @mav for his contributions to the DKG discussions.
We have spent some of the last few days devising an asynchronous Concrete Coin that we call: “gradient leadership based Concrete Coin”. We think that it exhibits all the properties needed for a Concrete Coin, but we removed all the synchrony assumptions from the process of obtaining it. We are in the process of formulating our suggestion precisely in our RFC, with associated mathematical proofs.
We are now tackling the second main challenge of mapping the ideas from “Byzantine Agreement, Made Trivial” to asynchronous consensus: integrating the Concrete Coin in the main protocol in place of a Common Coin.
We only started focusing our energy on this recently but are already seeing some promising leads.
While all this progress is being made on the algorithmic front, we are also continuing with our efforts of producing an RFC to explain the complete idea as clearly as possible, along with visual illustrations that should help convey our points more easily.
If and when we find a fully proven solution for the second challenge explained above, we will focus all our efforts into finalising the wording for the RFC and share it with the community.
We refer to this new adaptation of BOA as: Total Record of Incidents Via Interconnected Agreement Lattice (TRIVIAL) as an acknowledgement to the paper: “Byzantine Agreement, Made Trivial” that gave us the idea of trying to solve the problem without the need for a Common Coin.
Crust
This week we mostly worked on the netsim and tokio-utp crates. netsim is currently able to only parse and construct TCP packets. We extended it a little to recognize flags (control bits) and TCP options, e.g. selective acks. Also, IPv6 is a first class citizen in netsim and now it is finally supported. uTP is another tough cookie and we are still working on graceful connection shutdown issues. In the meantime, we managed to setup CI for the tokio-utp crate which lead us to discovering new bugs. In some cases, our uTP implementation sends Reset packets back, e.g. when the connection limit is reached and we receive a new Syn packet, or when some random packet with unrecognized ID comes, etc. It turns out the Reset packet is not always correctly sent out. Although, it’s worth to mention that the fix is on the way While doing various fixes to our uTP implementation, we raised some issues on the specification: BEP 29: packets with unrecognized IDs · Issue #83 · bittorrent/bittorrent.org · GitHub, BEP 29: graceful connection termination · Issue #84 · bittorrent/bittorrent.org · GitHub. Hopefully, the rest of the community will be able to make a use of our discoveries too.