With our tight-packed MaidSafe nodes starting to keel over as they filled up and ran out of memory, we waved goodbye to RewardNet a couple of days ago, but not before it had lived up to its name. As well as demonstrating the now familiar stability, community members were enriched by unimaginable amounts of imaginary money, and we were all rewarded with greater knowledge about some important topics. As always - a thousand thanks for everyone who helped out. Special shout out to @josh for his metrics on the spread of payouts across his nodes, @southside for diligently tracking prices, and @loziniak for the recent PR cleaning up certain client APIs.
Here’s what we have changed this week in response to the testnet findings.
Price change tolerance is better handled by the client than by the node. Nodes should not accept an underpayment, it’s up to the client to decide its tolerance to a price rise. The client can now repay if a chunk price changed between the initial query and file upload (remember, prices increase automatically as nodes fill up).
We have changed the price-capacity curve so that storage stays cheaper for longer, only really ramping up when nodes are half full. This will encourage usage and also smooth out divergent payments in the early days.
We have started paying for storage using UXTOs sent over the wire instead of full DBCs, which uses much less memory. Another bonus: UXTOs are encrypted in motion.
To deal with the issue of clients running out of memory when encrypting large files, we have added streaming capabilities to self-encryption, meaning that large files are chunked stepwise rather than all at once. We’ve done something similar with decryption on the network side too. And we have made concurrency tweaks to the client, which should hopefully help keep large uploads moving as they can send more chunks to the network at the same time.
Away from the testnet, there’s good news about Sybil resistance in general. As David mentioned last week, Sybil attacks shouldn’t be an issue for Safe Network on more than the petty vandalism level; but @bogard brought to our attention a paper that shows how Sybil attacks on other decentralised networks can be detected in real-time and potentially mitigated, which, if it works out, would be a real step forward for the wider decentralisation cause.
General progress
@anselme has now integrated UTXOs in preparation for removing DBCs from the network, including adding security checks and implementing chunk payments with SNT. Register payments will follow shortly - as will a full explanation of the advantages of this approach over DBCs (promise). One thing to note right now is that UTXOs are fully encrypted over the wire, whereas DBCs were not - a significant security improvement.
Payments for registers is currently @bochaco’s domain, and they are pretty much there now. He has also introduced UX improvements to prevent a client paying to create a new register if one already exists at the same address, and notify the client when errors have been made in the transaction or in the creation of a new register.
On monitoring, @roland has worked on integrating the OpenMetrics format into safenode
, and setting up Prometheus and Grafana to visualise the metrics that are emitted. This is a much lighter solution than we were using before.
Meanwhile, on downloads, @qi_ma has implemented a stream decryptor for downloads to allow the downloaded file to be streamed directly to a given folder. This avoids having to hold the entire decrypted file contents in memory before writing to disk, which should ease some mem problems.
@Joshuef has been looking at node pricing costs, and economics, including the price curve described above. Again, a fuller post will be forthcoming before too long. He’s also been getting stuck into moving payment tolerance handling to the client, so payments that fail due to price changes just get retried at the new rate. And he’s been looking into some weird intermittent CPU behaviour on Linux, which may be due to DBC loading.
@chriso has been looking into fixing a file sharing error and setting up automated smoke tests and initial data uploads for testnets.
@dirvine has returned to the ants’ nest for his latest blog post on the folly of trying to impose total order on complex systems.
And @jimcollinson has come up with some excellent ideas for a no frills, MVP network for launch, and proposed ideas for economic incentives in the beta network. Hate to do this to you again, but more soon!
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; German ; Spanish ; French; Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!