We continue to refine features ready for the next testnet, notably introducing pay-per-chunk and UTXO.
Paying per chunk means treating each chunk individually rather than the client bundling them into a merkle-tree and asking nodes for a quote on that file. Now clients query each node in the close group for the price of storing each chunk before sending it (whereas previously we attempted to get a price over the whole network, which was quite inaccurate), and pay the nodes returning the chosen quote directly. The nodes send their public key with their individual quote, and clients pay to that key. As mentioned last week, with the current group size of 8 the client should choose a price that guarantees at least 5 nodes will store the chunk.
Payment details including DBCs are now sent with the chunk, rather than a spent-proof as before, with chunks being stored only if the given node has been paid.
Operating at the per-chunk level is more granular than at the per-file level, which should smooth out transactions and allow for more precise payments to nodes and more accurate auditing. This also makes it easier to begin implementing rewards, as it should simply be saving DBCs sent with new PUTs at the node.
Which brings us neatly on to UTXO (unspent transaction output) which is a model a bit more like bitcoin, where the spentbook is stored on the network. Also, BLS one-time keys are used to de-link the owner from the transaction and prevent double-spend. Refactoring DBCs to incorporate these features is ongoing. @anselme and @bochaco, who are the ones deepest into this, are currently away, but we promise a full write-up when they return.
Bug fixes
Slow client connections have been fixed, as have a few testing issues with logging and file uploads and downloads. We’ve sorted the issue of reuploads reusing incorrect cached payment proofs for the moment by eliminating the caching stage – we’ll reintroduce that at a later date. And we’re now verifying data is copied to a majority of the close group before storing it to reduce storage errors.
General progress
@joshuef has been driving the pay-per-chunk changes, which allows attaching payment info to each chunk. He’s also fixed some bugs around this caused by attached payments changing the file size.
@Qi_ma is focusing on resolving niggling problems with testing and benchmarks related to file uploads, network communication and test reliability.
@roland is optimising how record distance ranges are set during replication and looking at fault detection to identify and reject invalid records.
@bzee has been implementing more connection debugging, and trying to find the source of the delayed join (and PUT receipt) some folk were seeing in the last testnet.
@chriso is implementing faucet as a service for testnets as part of his work into automating testnet deployment. He’s also extended the testnet inventory report to provide peer IDs to help bootstrap connections.
The travails over QUIC and libp2p
continue to occupy @bzee, who has also investigated reasons why unroutable peers remain inactive and don’t get dialled/added by other peers.
And @anselme has been researching and thinking through the evolving transaction system.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; German ; Spanish ; French; Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!