Update 28 September, 2023

This week we’ve been looking at the results of the testnet and working on fixes for bugs. The first thing to note is that, to make debugging easier, the testnet was deliberately unforgiving, with a chunk having to be replicated to all eight close group nodes to be considered valid. That said, it did unearth some strangeness around missing chunks which @qi_ma and @joshuef have been digging into.

Downloading large files was occasionally failing due to one or more chunks being missing from the download. Thanks to all who reported this. One reason we suspect for this is to do with caching. When fetching a record from Kademlia we have some choices. Quorum::One means we take the first reply we get, Quorum::All means we wait for all answers to come in and check that they match. Since Chunks are self verifiable (content addressed) one answer should be enough as we can check its validity on the spot.

However, it seems that Kademlia caching, which should cache chunks at closer nodes when using Quorum::One, does not work in quite the way we thought… It appears to only ensure one node holds the data, only (as opposed to still ensuring it goes to all data-holders, even though we only require one copy back). So we’re disabling that for now and reverting to Quorum::All to see how we get on.

Incorrect store costs are another possible reason for missing chunks. Sometimes the client requests store costs from nodes different from those that end up storing the chunks. This results in the storing nodes not being paid. We are also seeing some replication failures so we’re looking into those too. (This may well be related to the Quorum / caching work above!)

To make life easier, we’ve made it so that if one chunk fails to download, the whole process stops with a MissingChunk error message, rather than waiting 'til the end. We’re also improving logging to debug each batch of uploads and downloads. And since logs are providing valuable debugging info, we’re now logging output from clients and nodes by default. The logs are quite verbose, so be aware small instances will likely fill up quicker.

And we have added hardcoded bootstrap peers into the node and client code, so no more having to set the SN_PEERS variable.

Batch-size and concurrency seem to have some effect, with larger batch sizes significantly speeding up downloads and larger concurrency settings, up to 40 or so, doing the same. We’re now experimenting with the effects on performance of reducing the Close Group size from 8 to 5, which should bring quicker downloads and lower memory usage.

Thanks as usual to everyone who got stuck in and put the thing through its paces. As a reward, you got to enjoy the fun of living in a hyper-inflationary environment, plus one or two folks got extremely SNT-rich. Don’t spend it all at once guys.


CashNotes are the new name for DBCs that better reflects the way payments are actually made. The underlying code for it has not changed here.

Essentially, they are a local representation of tokens in a wallet. They can be spent on the network in exchange for new ones of the same value (total tx input/output value) by the recipients.

CashNotes are created in transactions and assigned to derived public keys. The derived keys are created from the recipient’s public key plus a random index. A different derived key is used for each transaction making each CashNote unique and unlinkable to the owner’s original public key.

The recipient needs the secret random index along with the parent transaction information to generate the corresponding derived private key in order to redeem the CashNote to receive payments.

General progress

@joshuef and @qi_ma have been the main team members involved with looking at incorrect store costs, failed replication and missing chunks. Josh raised a PR to fail fast as soon as chunks are missing during download, and temporarily removed Kademlia caching and switched back to Quorum::All to address the missing chunks issue.

As well as helping with the debugging, Qi continues to research libp2p caching to understand it better, and studied the GossipSub pub/sub implementation that @bochaco is working on too. That is now at the testing stage, and they are tracing how messages propagate between nodes. A little way to go yet on that. @bochaco has also been working on rewards notifications in the node.

@Anselme cleaned up unused code in the sn_transfers crate, minimised security risk by making modules private, and rewrote benchmarks using high-level transfer APIs to handle this change.

@bzee is working on reusing transfers when node costs change. When the storing node increases its price between the time the client requests a store and when it makes the payment, that payment is insufficient. Rather than having to start again, we want to retry with the original CashNote, then if that fails again top it up with an additional CashNote, which is quicker.

@Chriso Added support for versioned binaries in the automated testnet deploy tool and has been working on getting that running, alongside some nice improvements to the safe ux.

@roland also worked on fixes in response to the testnet findings, and @dirvine reduced Close Group size from 8 to 5 to improve performance. David has also been doing some further thinking on a secure upgrade mechanism for Safe.

Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!


first now to read :slight_smile:

great stuff thanks for all the work team and looking forward to the next testnet :slight_smile:


Second (For the first time). Good progress team.


I’ll have to take third then :partying_face: keep it up maidsafe team :clap:t2:


Thanks so much to the entire Maidsafe team for all of your hard work! :horse_racing:


Also, many thanks to all of our volunteer testers! :woman_factory_worker: :man_factory_worker: :factory_worker: :man_factory_worker: :woman_factory_worker: :factory_worker:


Thx 4 the update Maidsafe devs

ClientImprovementNet was really a joy to test, especially downloading/viewing 2GB crow.mkv

Hyper-inflation is sure fun, I paid 241.080098248 SNT for 1 chunk, luckily I could print more currency “out of thin air”. :money_mouth_face:

Keep hacking super ants

Now test new testnet :partying_face:


CashNote! I like it! And upgrades, oooohwee.


Nice update team. Thanks as usual for everyone’s efforts.

That’s good to know! I confess I was a tad bit worried.

In terms of ‘terms’, what is the relation between CashNotes and SNT? Or are they sort of the same thing?

Is libp2p caching a separate caching layer to Kademlia or are you talking about the same thing here?

Now to go check out the current testnet! :man_bowing:

Cheers :beers:


cant come soon enough!


How about the current testnet, is there some things that hinders it’s performance? Not complaining, just curious. :slightly_smiling_face: j


Thank you for a comprehensive update - and thank you for yet another testnet coming along so soon after the last one.
Cant help feeling its all getting faster and faster and closer and closer :slight_smile:


Not so much, we are just adding more eyes on the prize, really. We will continue to be super aggressive in the short term.


I can’t wait to get a big bunch of CashNotes.



Thank you for the heavy work team MaidSafe! I add the translations in the first post :dragon:

Privacy. Security. Freedom




… with my mind on my CashNotes and my CashNotes on my mind.

(not a dev)


Great work MaidSafe, and always nice to see a new test net up and running!

To clarify, why would we expect faster downloads if there are fewer places to download from (5 vs 8)?


It’s more a matter of how many places hold a chunk. Then pick one and get it. Over all chunks for a file this will mean parallel downloads (like bittorrent). The applicant count is really how many nodes NEED to hold the data so that it does not vanish with churn.

So we have close groups (5) but we will also have

  • Caching
  • Archive nodes.

So adds in more locations for data, albeit in.a more chaotic manner.

This relates to doublespend protection in SNT. So we have this process to validate a transaction

  • Is the parent SPENT (i.e. is the parent of this transaction spent to YOU and signed by the parent)

So this is fundamentally, it is as simple as that. This is where we deviate from full consensus (btw there is no such thing as partial consensus networks, it’s all or nothing and all is unnatural).

The crux is the above. Did a valid key spend ONCE to another key that has never been seen before?

The attacks consensus etc. try to avoid is this. Is the network telling us the truth?

Most networks use a full consensus mechanism where they get every (or some form of majority) node to agree on that single source of truth etc. All sounds great, get the world to agree on valid FINAL transactions.

If we split the transaction into 2 parts

  • The Parent was SPENT exactly ONCE
  • The CHILD has never been seen (never SPENT)

Then a transaction is the PARENT part spending (giving to a child) and then the CHILD spending.

You can think of a doublespend ATTEMPT as a PARENT spending the same token to more than one CHILD.

Remember (ATTEMPT not a SPEND)

So to facilitate a FINAL doublespend the attempt has the be hidden and then the more than one CHILD needs to SPEND as well.

It ultimately boils down to ensuring the PARENT is spent exactly once.

What we do is store the PARENT spend to the close group of that parent. This is where folk get nervous, that’s not the whole network or even 2/3 of it.

Now introduce chaos

  • Sybil defence of close group make them difficult to target.
  • Separate CHILD attempted payments will query the close group (if they are bad what CHILD do they say they spent to?)

So some doubt and worry (but not much). Then add in the following (gossipsub comes into place again, just like archive nodes)

  • People will run a gossipsub service to accept all payments in some part of the network

  • A node, any node can subscribe to that to see transactions

  • Other folk will run a gossipsub service purely to accept doublespend attempts (remember PARENT must have signed for more than one version of a transaction, so on purpose)

  • Likely every node will subscribe to get these alerts

So we have

  • the close group security to hold correct PARENT transactions
  • Service providers collecting all transactions for part of the network (random folks)
  • Service providers broadcasting double spend attempts to everyone

This is the chaos I mention. We cannot tell who could provide these services and cannot stop them. Nobody can.

The key point being a doublespend attempt is not a doublespend, but it is intentional. It cannot be forged, It can be broadcast though and it is verifiable. So we can broadcast that in many many ways. Projects and people with a vested interest in the security of the currency will likely run such services, they will also make use of them for their own project, to gather intel on transaction capability etc. Maybe to even audit the supply, lots of reasons.

The vision of a transaction being a 2 step process (it always is) and seeing that an attempt is not a spend, but an attempt that is visible BEFORE and actual spend happens is quite powerful. It’s also incredibly simple. This is why we can parallelise the currency almost infinitely. It’s a simple check PER TRANSACTION (not a whole chain of blocks of transactions)

I imagine there will be a slew of new hyper speed nano transaction networks in the future, but for now this is our foot on the ladder. I love the chaos part and the non consensus / total order requirement. It makes for simple code and gives potential hackers a nightmare scenario. It makes you wonder how many other projects and systems could benefit from this approach (AI does, ANN’s certainly do)


A key point here is this

  • If you attempt a doublespend - you will lose your cash

This seems scary, but it’s not. A doublespend attempt is intentional.

A common negative folk mention is this is unfair. What if a wallet was badly programmed, etc. But I think that is not a good reason to not kill such behaviour. Badly programmed wallets should not exist, and there is much more wrong they can do that try and doublespend. So they just need to be killed off and not used, like any piece of faulty software.

A thing to watch out for is this :

You get paid X from some dude.
The dude has your invoice, you got paid, all is well.

Then the dude says I don’t want you to have the cash so I will just do a doublespend. The network detects the doublepspend attempt and kills the funds.

This is the downside here. When you receive the money you should send it to yourself again (wallets should do this automatically). If you do this, then the bad dude still kills the PARENT but you have already spent the CHILD. So all is well for you.

So the bad due tried to cause you harm, but fails.