For the next testnet we hope to put to bed the memory issues that sank the last one, so that’s been our focus this week.
The pay-one-node branch is looking good, and with a bit more testing will be ready to merge. It should reduce memory usage compared to the previous approach of paying all nodes that store a chunk. As we mentioned last week, ultimately we should be able to offer upload speed and assurance trade-off options in the client.
Paying just one node also simplifies the royalty payments. Compensating five nodes means five separate royalty messages being broadcast for the same chunk, which then need to be deduplicated by the receiving nodes. This is complicated by the fact that these CashNoteRedemptions
are encrypted. Using one node simplifies this a lot. In addition, these gossiped messages no longer need to be encrypted (with no loss of security) so we’re putting that in now.
The exact impact on royalty payments memory use needs to be tested further on a larger testnet, but it looks like some other simple optimisations are possible too. For example, setting each node to subscribe only to relevant messages, not all messages, has seen a big reduction in mem, and we’re looking to stop unnecessary rebroadcasting too.
On GossipSub
itself, we’ve made verification stricter for testing. Rather than just expecting a minimum number of messages now we check for the exact amount.
@southside’s PR to improve logging outputs is now in the codebase. And thanks also to @loziniak for a PR to clone ClientRegister
.
Now we’re going to hand over to @anselme to walk us through how payments work.
Payments on the Safe Network
This is a summary of how we pay for data on the Safe Network as well as an introduction to money transfers in general.
Transfers
Before we get into data payments, let us first understand how basic transfers work. Everyone on the Safe Network, both clients (wallet apps, Safe CLI, etc) and nodes (data storage) have a public/private keypair. The public key (aka MainPublicKey
) is used to receive tokens. It is equivalent to bitcoin or ethereum addresses. The private key (aka MainSecretKey
) is used to sign transactions and prove ownership of money.
This is what a MainPublicKey
looks like:
93d01b3c6c0d41c4e50c855c753f906ba478f97a838415e6d74615a4b037a7101e724f935727bbf23d17293ab74a3027
You can obtain your own with the command: safe wallet address
Money in a wallet is stored as CashNotes
. These are like a cheque for a given value. They contain all the necessary information for the owner to spend the value in them, but don’t have value on their own as they are useless without the private key.
When someone wants to send money, they create a Transaction where they spend their own CashNotes
in exchange for new CashNotes
that are owned by the recipient. As those CashNotes
contain secrets that might affect the privacy of the sender and the recipient (the MainPublicKey
), they are never sent or stored on the Network.
Instead, the sender creates very small CashNoteRedemptions
that contain the minimum necessary information that the recipient needs to verify the transaction and re-create their own CashNote
on their side.
Those CashNoteRedemptions
are encrypted and packed into a Transfer which is sent directly to the recipient. The recipient of the Transfer can then decrypt it using their MainSecretKey
, verify and rebuild the CashNotes
, before adding them to their wallet. At that point, the transfer is complete!
You can send a Transfer with: safe wallet send <amount> <to>
Example: safe wallet send 42 93d01b3c6c0d41c4e50c855c753f906ba478f97a838415e6d74615a4b037a7101e724f935727bbf23d17293ab74a3027
This will print out the encrypted Transfer that can be published or sent directly to the recipient.
They can receive it with: safe wallet receive <transfer>
Data Payments
Data on the Safe Network is paid for using those same Transfers.
Before data is stored on the Network, it is prepared locally by the user: cut into chunks and encrypted. Each of those chunks have a unique hash value from which we derive their NetworkAddress
.
Nodes on the Network also have a NetworkAddress
. Data is stored on the nodes which have the closest NetworkAddress
to the data itself. This is why we say that data is content-addressable.
By querying the Network, a client can get the closest nodes to a NetworkAddress
. Once we know them, we first perform a StoreCost
query to ask the price for holding data at that location. The price depends on the storage capacity of the nodes, which ultimately depends on demand.
Once we know the StoreCost
we can safely send the data along with the Transfer paying for it. Upon receiving the data, nodes verify the Transfer, take the payment, verify the data itself and store it.
Along with that Transfer, another small Transfer is sent: NetworkRoyalties
, These are sent to a pool and help keep the Network alive. When receiving data payments, nodes make sure that those NetworkRoyalties
are also valid before storing the data.
All of this is done automatically when you upload data using the CLI:
safe files upload some_file
General progress
@chriso has finished work on a promising WinSW service manager extension to help with node management for Windows and submitted a PR for it.
@roland Submitted a PR to resume CLI uploads across multiple runs. This stores chunks locally and moves/removes them once paid/verified. He also refactored the codebase to release RPC client as a binary to fix some testnet deployment errors.
Several team members have been looking into the mechanism of paying royalties to Foundation nodes, as described above, including @bochaco, @bzee and @qi_ma.
@qi_ma also created a PR to avoid duplicated hash work, making the most of the latest changes to libp2p.
@jimcollinson has been looking at what’s needed for an MVP launch, with a focus on on-ramps and off-ramps for a soft launch.
And @joshuef has been drawing up a priority list of the remaining technical work before we enter beta.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ;
German ;
Spanish ;
French;
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!