Our primary goal with Safe Network is data permanence. We store copies of data on devices that statistically are unlikely to be in the same region should, for example, a massive power cut or government-imposed internet take-down occur. For this purpose, the more copies the merrier, but that imposes demands on storage and, in particular, network bandwidth. This week we look at one approach to tackling this issue, which should allow more redundancy with less data flow.
General progress
@Anselme has fixed a consensus bug, implementing anti-entropy for handover as well as generations. More comprehensive join retries (where we ensure weāre in the current section state) have been merged too 
.
@Chriso is digging into DBC implementation and has already completed all the CLI changes for passing a DBC to a new owner. Heās now turning his attention to the sn_api side for assigning the change DBC owner.
We can also report progress on the observability front with @yogesh making nice headway with the traceroute messages, making sure all the message flows (file and register) have a traceroute response back to the client from the adults and are logged in the tests.
And @davidrusu has been spreading the word, attending another Toronto CompSci meetup to talk through Tree CRDTs. āA great discussion all aroundā, he reports, and a great place to recruit interested parties to Safe Labs.
In the wider safe-world, the ongoing community project to support MaidSafeCoin on the ERC20 protocol continues unabated, and we are please to announce these efforts continue to bear fruit! So in addition to Uniswap eMAID is now also open for trading on P2PB2B!
You can thank the dedicated community members such as @Sotros25 and @anon61899651 who have taken it upon themselves for pushing this topic forward continually 
Optimising data transfer
When someone makes a GET request, that kick starts a number of processes. First, the request is relayed to the elders in which section the chunk is stored. The elders then calculate which four adults store the chunk - the four XOR-closest to the chunkās address - and each of those adults pass their copy of the chunk back to the elders, who then return them to the requesting client.
This provides redundancy, there are always four adults with a copy of the chunk, and also gives us a way to monitor performance. If one adult is significantly slower or less reliable, it can be demoted. But it comes with a price tag.
If the client contacts all seven section elders asking for a 1MB chunk āAā, and each elder in turn requests chunk A from the four adults holding it and returns them to the client, then potentially thatās 28MB (4x7) passing across the network for a single 1MB GET request.
Currently, we are using a halfway-house system where the client contacts three elders at random and those elders contact the four adults holding chunk A - so we potentially have 12MB (3x4) of data flowing across the network for a 1MB request - better, but still not great. And reducing contacts to three elders comes at a cost: if we have only three elders interacting with the adults we no longer have a supermajority to decide if one of the adults is dysfunctional, making dysfunction tracking more complex.
A solution we are looking at now is information dispersal algorithms (IDA, also known as erasure coding). This could help us significantly reduce the volume of data transfer per GET, by having adults pass a share of the data to the elders rather than the whole thing. The elders then pass these data shares back to the client, who reassembles them and, voila, chunk A is recreated. Potentially this could reduce the flows on a GET to just 1.4MB for a 1MB chunk.
So how does it work? First things first, weāre not proposing replacing replication with an IDA. Some projects do this (e.g. Solana) but for us there are too many tradeoffs. So, just as now, if an adult goes offline its data is replicated in full to another adult. The change is that adults will be able to split chunk A into seven pieces using an IDA and pass just one piece back to the elders on request, rather than the whole chunk, meaning that much less data is exchanged.
Once it has collected five out of the seven pieces, the client can reassemble chunk A.
This five-out-of-seven figure doubtless switched on a few mental lightbulbs 
, because itās the same threshold that we use for BLS consensus, but IDA is a different beast from threshold cryptography, designed for optimising data storage and transfer rather than secret sharing. In case youāre wondering where the extra 0.4 comes from (transferring a 1MB chunk using IDA requires 1.4MB of data flows), this is an unavoidable overhead from the way the algorithm works.
So, less stress on the network and the client too, and of course all seven elders are now in contact with the adults, making consensus over dysfunction much simpler.
Further optimisations should also be possible with this architecture too. Because of recent Membership changes, we now know which adult of the three holding chunk A is actually closest. So rather than asking for chunk A, the client can ask it directly for the first piece A[0] and the elders will go straight to the closest adult; the second piece A[1], the elders will ask the next-closest adult, and so on. Any failures are simply retried at another adult. This is more efficient and means we can potentially increase the number of replicas - adults holding chunk A - without any significant extra stress on the network, with obvious gains for data redundancy.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, weāre always looking for feedback, comments and community contributions - so donāt be shy, join in and letās create the Safe Network together!
awesome
Southside! Beer Me!
, in this case 
, such that 
and 
.
