More than anything else, the thing that makes reliable decentralised and distributed networking difficult is asynchronicity: we just can’t guarantee that message A will arrive before message B, or even if it will arrive at all. With a few nodes that’s OK, but in a section with tens of nodes the number of possible event and messaging combinations is huge, and a real challenge to reason over. This week we take a look at Stateright, a very useful tool for testing our algorithmic assumptions since it tries all the different possibilities.
General progress
@bochaco is testing the network’s response to different chunk sizes, including how it handles data maps. Small chunks are faster to transmit, but they make for much larger data maps, so we have to recursively self-encrypt them until they are less than chunk_size. There are other rather technical trade offs too, but basically we’re looking for an optimum size for our chunks.
@anselme and @dirvine are looking at a reason field for DBC SpentProofs - the proof of a transaction. This is a new field that contains the reason for the expenditure. It could be the hash of chunks paid for or a purchase order number, or some other sort of label. The important thing is the elders will be able to tell if it is valid or not (for example, if it is payment for a single chunk, does the Tx match the storage price for the data?).
Why do this? Simplification and security. It could mean that we can do away with the need for elders to sign data. So, no more risk of ‘old key attack’ where if a bad actor gets hold of a previous key they could use it to validate data. With this design, old elders’ signatures will no longer be valid.
The flow:
- Client asks for quote to store data: 1 SNT please
- Client reissues a 1 SNT DBC for the elders with a reference to the data as DBC reason
- Client tells elders it is spent by sending SpentBook entries for that DBC (the entries contain the
reason) - Elders check that reason and update their SpentBooks
- The DBC is now officially spent for reason
- Client asks the current elders for a signed copy of that SpentBook entry
- Once client gets enough signatures, it can store the data by providing a current-section signed copy of the SpentBook entry along with the data
- The elders check if the section signed (meaning most elders see the same reason) SpentBook entry refers to the data, and stores it.
We continue to remove read/write locks in the code for better performance and simplification (no multithreading unless absolutely necessary). @joshuef is working to remove the big one around the node state itself.
As some sharp-eyed community members have already spotted @oetyng is modelling a simplified version of a payment network.
@chriso is working on improving aspects of the CLI including reworking the update command.
And @davidrusu has been getting stuck into Stateright.
Stateright
Stateright is a Rust library that promises to be a big help in testing assumptions. Basically, it’s a model checker that runs through every possible combination of events to check whether a stated assumption is valid.
Decentralised networks are hard to model for a variety of reasons: nodes can die, nodes can be Byzantine, messages can be delayed, messages can arrive out of order or not at all, you can get unexpected race conditions, etc, etc. Stateright helps us work through all those possibilities.
As an example, we might want to know whether DBC doublespend is possible with our current design. We model the assumption “Doublespend is not possible” in code, add that to our existing code base and set Stateright on it. Stateright will then run through every single combination as allowed in our algorithm model. It will delay messages, make two elders fail at once, and send events out of order. Once it has exhausted every possibility without being able to doublespend, we get a nice green tick - our assumption is valid. Every state it found was right.
If something fails, it optionally redirects us to a GUI where we can step through the messages that led to failure and work out what the problem was.
The great thing is, it tries everything, not just a subset of possible combinations. On decentralised networks it’s the edge cases that kill you, and with Stateright, unlike other methods which use sampling, you can be sure it’s covered them all. There may be millions of combinations to run through but so long as we’ve written the assumptions properly it is pretty quick.
The other thing is that it is a big help in iteratively correcting our code. It is a few extra functions we can put in our production codebase to check models. We will most likely use this to write specific pieces of production code and do so in a model. When the model is satisfied we take that production code and get it into Safe.
You can find out more about Stateright in the docs or from this video - which even though it’s an intro is quite technical.
Better still, here is a link to our current experiments. For the nosy/interested/curious just clone this repo GitHub - maidsafe/stableset-experiments. We use the master branch from Stateright, so you will need to also clone that in the same folder where you cloned the experiments repo.
Be aware, though, this is real-life Safe Labs experimental data and changes significantly day to day as we up the ante in our searches for simplicity.
If you do a cargo run --release and then open http://127.0.0.1:3000 you will see the GUI. You can then manually click what messages to send or indeed click run to completion and it will show you where any current issues are. Be aware we almost always have issues there as we are iteratively testing, so don’t feel disheartened, it’s actually great 
.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
what we think does pan out.