Today, we were planning to release new versions of the SAFE Browser and the example applications which have been updated to work with the latest API changes. The changes are mostly related to the safe_app/DOM APIs and also to improving the developer experience.
However, there is a limitation in the browser which we are working on fixing at the moment. If we open a web page, say www.example, it loads fine, however, www.example/index.html is not being loaded. We will be releasing new versions of the browser and the example applications as soon as this issue is resolved (tomorrow hopefully).
As we mentioned in a previous update, we would keep you up to speed with developments in our new Indian office. Circumstances have enabled us to take a bigger office for the same monthly costs within an IT-specific business park. The additional square footage will facilitate up to 30 staff as opposed to the previous 15. The one caveat is that the bigger office space requires more internal work and will therefore push out the anticipated opening of the office to late January, however, we felt that the bigger space was worth the slight delay. With the front-end team starting to make regular new releases, there is much to be excited about bringing this team closer together and growing its capability even further.
SAFE Authenticator & API
SAFE App Node.js and SAFE Web API have been updated to adapt to recent API changes in SAFE Client Libs, summarised as follows:
SAFE App
Addition of following functions to app object in safe_app_nodejs and window.safeApp in web API: isNetStateInit, isNetStateConnected, isNetStateDisconnected, clearObjectCache, isMockBuild.
Crypto
Addition of a signing key API on app.crypto object in safe_app_nodejs and window.safeCrypto in web API: generateSignKeyPair, pubSignKeyFromRaw, secSignKeyFromRaw, generateSignKeyPairFromRaw.
signKeyPair.secSignKey.signorwindow.safeCryptoSecSignKey.signto sign data on the networksignKeyPair.pubSignKey.verifyorwindow.safeCryptoPubSignKey.verifyto verify signed data on the network- Note that the
safeCryptoKeyPairobject is replaced bysafeCryptoEncKeyPairandsafeCryptoSignKeyPairinsafe_app_nodejsandwindow.safeCryptoKeyPairis replaced bywindow.safeCryptoEncKeyPairandwindow.safeCryptoSignKeyPairinweb API.
Mutable Data Permissions-set
Removes MutableDataPermissionsSet and window.safeMutableDataPermissionsSet.
Mutable Data Permissions
forEach changed to listPermissionSets, which now returns an array of permissions represented as strings, instead of the no longer existent permissions-set object.
Mutable Data
- removed free function
- removed newPermissionsSet function
- getKeys returns array of mutable data keys
- getValues returns array of mutable data values
Mutable Data Keys
Removes MutableDataKeys and window.safeMutableDataKeys.
Mutable Data Values
Removes MutableDataValues and window.safeMutableDataValues.
It’s suggested to browse documentation here to learn the APIs:
And to read the tests for each API to view examples of how each function works:
We have an issue with the loading of certain URLs in SAFE Browser. We are working on addressing this issue. We will be releasing the browser and the example applications as soon as this issue is resolved (tomorrow hopefully).
@joshuef has been working on the custom browser over the last couple of weeks and has made considerable progress. We are testing the initial version of the custom browser which makes use of the latest APIs and allows safe:// only browsing. The UI is still naive as the focus was on functional aspects.
A pull request for desktop support for the Csharp APIs has been raised and is currently under review. We do hope to get it merged once it is tested. @rachit has been expanding the APIs and test cases. @joy is beginning the integration with the Java JNI bindings provided by the core team. We are hoping to get the Java API covered as soon as possible for Android and desktop platforms.
SAFE Client Libs
We’re nearing the completion of the binding generator for Java and C#. We’re starting to work with the front-end and mobile teams to collect their input and their requirements for the bindings, and so far we’ve gathered some useful feedback on the C# bindings. With that feedback, we intend to cross off most of the TODOs during this week and start to integrate it with the actual .NET and Android/Java projects.
@marcin has completed the automated tests for the Alpha 2 network, which we mentioned in one of the past updates:
These tests already helped us to uncover some obscure bug which we previously didn’t encounter, it concerns the apps revocation on the Alpha 2 network using the current master branch build of SAFE Client Libs. We’re actively working on pinpointing the exact cause of this bug and fixing it.
Routing & Crust
As data chains discussions and designs are nearing finalising phase, we decided to start coding features to better express the fundamentals and continue testing the design. Work has started at last. We have branched off Routing master into dev. There one can now clearly see the fundamental types we will be using in the implementation of data chains. We’re currently working on a document defining and detailing the different parts of the changes that data chains integration brings and hope to have this ready and published on the Dev Forum next week (barring any nasty surprises early next week). In the interim, here are some of the key definitions that have been fleshed out in that proposal:
Complete group: GROUP_SIZE peers with age >4.
Elders: the GROUP_SIZE oldest peers in the section. If there is a tie, we use the tie-breaker rule for node aging.
Tie-breaker rule for Node Aging: If there are multiple peers of the same age then XOR their public keys together and find the one XOR closest to it.
Adult: a peer with age >4 in a section with a complete group.
Infant: non-elder with age ≤ 4.
Vote: a node’s detection of a network event plus its signature of it.
Proof: a node’s id and its signature against a network event.
Block: network event with a vector of proofs collected.
Group consensus: a block containing a quorum of proofs of the elders.
Section Membership: All the members of a single section, comprising the elders, adults and infants.
Neighbours: sections we are supposed to be connected to differing in exactly one bit from us.
Sibling: A section differing from us in the last bit.
Corresponding code is also being integrated into the dev branch, for instance here are the events that will be voted on. Once quorum or more have voted we form a block for the event with their proofs. Some of these blocks will go into the data chain. For context, here is a Vote which is exchanged between Peers directly. A Proof is extracted from a Vote to create a Block which gathers the Proofs and when it reaches a quorum of Valid voters the Block becomes valid. These valid voters are deduced from the chain order.
As we had stated earlier, in a section we will usually have GROUP_SIZE elders, some adults and some infants. Basically, elders are the oldest nodes in the section and it’s their votes that matter. If an elder is lost, an adult is promoted to take its place. Infants are the ones whose age is <=4 so they must stay long enough to see enough churn and get relocated for them to become an adult. Until then they do not contribute towards churn. This is because such young nodes have a much higher probability of coming and going offline thus the network does not rely on these nodes for general operation.
With that brief categorisation in mind (much more detail will be available in the document we are currently formulating), the (local) events witnessed by a section will be voted by the elders and when a block is formed as described above, we will decide whether that should make it into the data chain or a state-container. For e.g. if we were accepting some peer who is going to be in the elder group we put that block in the chain. If it was an adult or an infant we wouldn’t. This will reduce the size of the chain considerably because it’s elders who actually matter in the network in our current design. They are connected to the neighbouring sections’ elders etc. Infants can just come and go (because they have a small age, so they don’t have great incentive to remain online or they could just be spawned by casual/malicious users for appearing temporarily on the Network) and if we recorded all that, we would have ended up with an enormous chain. So one needs to “earn” one’s position in the chain - by becoming an elder and by that very virtue meaning that one has spent enough time in the network being a good node and also disincentivising them to act badly which would compromise their age which they’ve earned over a long time, few will wish to harm their nodes farming rate in that manner 
This is also a very important defense against any botnet type attacks, or large adversaries with boundless cash who may wish to attack the network in huge numbers. The network simply won’t allow such mass joining in short periods, but will instead ensure joining is a process the network controls through need. Therefore the network will require nodes when it needs them at the rate it needs them and upon accepting nodes, the network will monitor and validate the capability of the nodes (infants) over time as they mature and “grow up” to become valuable members of the network. A very natural design as inspired by the natural world, as always, if you cannot find your design in nature, it’s probably wrong. We are delighted we can and these steps are very powerful in their simplicity.
Now that we have started with the code, we also thought it would make sense to quickly run some simulations to see things like what is the section size before we split (because we’re currently hoping to only split once we know both post-split siblings will have enough elders and adults) so that if we end up with sections with a great many infants then we quickly devise ways to not have so. E.g. for a start we could say in a section with proper categorisation of peers into elders, adults and infants, don’t accept any more infant with age of 1 if we already have such an infant. @bart’s created a small simulation to test this and it requires some finishing touches but it has already given us some very useful results.
@anon86652309 and @qi_ma are working on finalising the detailed data-chains document which will have the definitions and explanations and also rule-sets for various scenarios such as merges and splits. This will include the message flow, which makes looking at the algorithm much easier so that it can be reasoned about or improved and finally put to code and tested.
In Crust, work has nearly finished with the integration of the p2p library. There’s still a couple of bugs that we’re aware of in p2p (e.g. it doesn’t close router ports once it’s finished with them and so ends up hogging all the ports), but we expect these to be simple to fix. Once those PRs are merged and tested we will be able to start integrating our uTP library as well and Crust will no longer be limited to the slow and peer-to-peer-unfriendly world of TCP. A significant step to a secured multi-protocol, randomised port encrypted network library that will benefit a great many projects.
hooooooooooooo
.
Got it! Thanks