RoyaltiesPaymentNet [Testnet 31/10/23] 🧛 👻 [ Offline ]

[maidsafe]

The objective of this testnet is to test that network royalties payments are made by clients as part of content storage payments, and that they are submitted to nodes when uploading the content.

Nodes are expected to not only verify that network royalties payments are valid, but they shall also be sending notifications through gossipsub for the beneficiary to subscribe to and receive the cash notes. In this testnet, our genesis node is expected to collect such royalties payments cash notes so we can check they are received and in good shape to deposit them to a local wallet, mimicking what the beneficiary of these royalties payments would be able to do once its public key is setup in the launched network.

Other interesting updates since the last testnet was launched:

• We’ve been tweaking the upload/verification process to speed things up (not blindly retrying).
• Nodes can now specify the number of log files to keep
• old hat hashing of content that was still pointlessly being done node-side has been removed.
• Node users can specify the metrics port if they are building for that feature.
• Various memory/replication optimisations.

Network Details

Node version: 0.96.19
Client version: 0.84.34
Faucet url: 138.68.153.228:8000

We have 101 droplets running a total of 2001 nodes. One droplet has 2vcpu and 4GB of memory.

---

If you are a regular user, see the ‘quickstart’ section for getting up and running.

If you are a first-time user, or would like more information, see the ‘further information’ section.

---

Quickstart

If you already have safeup, you can obtain the client and node binaries:

safeup client --version 0.84.34
safeup node --version 0.96.19

Run a Node

Linux/macOS:

safenode

Windows:

safenode

Check local node’s reward balance

Your local node’s peer id will be printed to the terminal on startup with an example command). (You can also retrieve this from the node directory.)

safe wallet balance --peer-id="<local-node-peer-id>"

Connect to the Network as a Client

Linux/macOS:

safe wallet get-faucet 138.68.153.228:8000
safe files upload <directory-path>

Windows:

safe wallet get-faucet 138.68.153.228:8000
safe files upload <directory-path>

To do this with non-default concurrency or batch-sizes (along with SAFE_PEERS set as above):

safe files upload -c 10 --batch-size 40 <directory-path> 

10 and 40 being the integer values you want to set for each

---

Further Information

You can participate in the testnet either by connecting as a client or running your own node.

Connecting as a client requires the safe client binary; running a node requires the safenode binary.

Obtaining Binaries

We have a tool named safeup which is intended to make it easy to obtain the client, node, and other utility binaries.

Installing Safeup

On Linux/macOS, run the following command in your terminal:

curl -sSL https://raw.githubusercontent.com/maidsafe/safeup/main/install.sh | bash

On Windows, run the following command in a Powershell session (be careful to use Powershell, not cmd.exe):

iex (Invoke-RestMethod -Uri "https://raw.githubusercontent.com/maidsafe/safeup/main/install.ps1")

On either platform, you may need to restart your shell session for safeup to become available.

Installing Binaries

After obtaining safeup, you can install binaries like so:

safeup client # get the latest version of the client
safeup client --version 0.84.34 # get a specific version

safeup node # get the latest version of the node
safeup node --version 0.96.19 # get a specific version

safeup update # update all installed components to latest versions

When participating in our testnets, it is recommended to use a specific version. In our project we release a new version of the binaries every time we merge new code. This happens frequently, so over the lifetime of a testnet, many new releases will probably occur. So for participating in this particular testnet, you may not want the latest version.

The binaries are installed to ~/.local/bin on Linux and macOS, and on Windows they go to C:\Users\<username>\safe. Windows doesn’t really have a standard location for binaries that doesn’t require elevated privileges.

The safeup tool will modify the PATH variable on Linux/macOS, or the user Path variable on Windows. The effect of this is that the installed binaries will then become available in your shell without having to refer to them with their full paths. However, if you’re installing for the first time, you may need to start a new shell session for this change to be picked up.

Running a Node

You can participate in the testnet by running your own node. At the moment, you may not be successful if you’re running the node from your home machine. This is a situation we are working on. If you run from a cloud provider like Digital Ocean or AWS, you should be able to participate.

You can run the node process like so:

# Linux/macOS
safenode

# Windows
safenode

This will output all the logs to the filesystem, with the location of logs being platform specific:

# Linux
~/.local/share/safe/node/<peer id>/logs

# macOS
/Users/<username>/Library/Application Support/safe/node/<peer id>/logs

# Windows
C:\Users\<username>\AppData\Roaming\safe\node\<peer-id>\logs

If you wish, you can also provide your own path:

# Linux/macOS
SN_LOG=all safenode --log-output-dest <path>

# Windows
$env:SN_LOG = "all"; safenode --log-output-dest <path>

The advantage of using the predefined data-dir location is you can run multiple nodes on one machine without having to specify your own unique path for each node and manage that overhead yourself.

Connecting as a Client

You can use the safe client binary to connect as a client and upload or download files to/from the network.

Using the Client

You’ll first need to get some Safe Network Tokens:

safe wallet get-faucet 138.68.153.228:8000

You can now proceed to use the client, by, e.g., uploading files:

safe files upload <directory-path>

To download that same content:

safe files download

This will download the files to the default location, which is platform specific:

# Linux
~/.local/share/safe/client/downloaded_files

# macOS
/Users/<username>/Library/Application Support/safe/client/downloaded_files

# Windows
C:\Users\<username>\AppData\Roaming\safe\client\downloaded_files

To download to a particular file or directory:

safe file download [directory/filename] [XORURL]

Troubleshooting

Cleanup

If you’ve used previous versions of the network before and you find problems when running commands, you may want to consider clearing out previous data (worthless DBCs from previous runs, old logs, old keys, etc.).

# Linux
rm -rf ~/.local/share/safe

# macOS
rm -rf ~/Library/Application\ Support/safe

# Windows
rmdir /s C:\Users\<username>\AppData\Roaming\safe
or
rmdir C:\Users\<username>\AppData\Roaming\safe -r -Force

If you encounter a problem running any of our binaries on Windows, it’s possible you need the Visual C++ Redistributable installed.d

[wydileie]

Another Testnet, awsome work. Unfortunately I will have to still sit this one out until the hole punching is working. Have fun everyone!

[Vort]

@maidsafe

network royalties payments are made by clients

What it means? Is it described somewhere?
How it differs from payments for uploaded data?

[JimCollinson]

Network Royalties are described in RFC 0061

[TylerAbeoJordan]

Uploading underway!

[aatonnomicc]

Nodes coming online uploads starting shortly happy Halloween

[Josh]

@aatonnomicc

Nodes coming online

I failed miserably and started my nodes without specifying the amount of logs to keep just going to live with it instead of starting over.

[Toivo]

Two nodes up, both use much RAM and increase fast. After 10 minutes 103 MB.

[happybeing]

Another test network another vdash update …

Vdash - Node dashboard for Safe Network

With vdash v0.13.2 you can: select a node in the summary list and go directly to its detailed view by pressing <enter> switch between summary and single node views by pressing <enter> sort the summary table by column (using and ) reverse the sort order with <space> I have also added display of node Process ID (PID) in the node status screen thanks to a suggestion from @storage_guy , and have improved display of earnings and storage cost. Feedback I’m intereste…

[Josh]

@Toivo

use much RAM

Scotland we have a problem,
Memory used: 174.004MB

[joshuef]

@Toivo

Two nodes up, both use much RAM and increase fast. After 10 minutes 103 MB.

@Josh

Scotland we have a problem,

Yes and no. Ram usage will be higher as there’s a whole 'nother set of things afoot here with the gossip layer for transfer propogation of royalties payments.

So ram usage is expected to be higher.

Now here, we’ve actually gone with a version that was testing ok but we’ve not waited for some upstream fixes that are going into libp2p here and here which drastically reduce the gossip mem usage (though I’d still expect us to exceed 100mb here, I think…).

It may be that this version won’t hold up for much usage with the current libp2p v. But I didn’t think it was worth holding up testing out royalties on mass for these fixes. (And indeed, it’s already looking like things are working on that front, which is nice).

Butttt, it may be if memory is still spiking despite our attempted workaround (which has reduced mem usage from >500mb/node already) that this becomes unusable fast.

We’ll see how things go! (And if we have to we can easilly disable royalties for now to continue on other test-paths, so it’s not really a worry there)

[Toivo]

@joshuef

We’ll see how things go!

That’s why we are here.

Uploading is very slow, has been about 15 minutes and a 60 MB file is still uploading. A lot of

chunks were not stored. Repaying them in batches.

Output in terminal is very unclear:

115 chunks to be checked and repaid if required
Verified 115 chunks in 285.283579299s
31 chunks were not stored. Repaying them in batches.
⠋ [00:03:25] [#########>------------------------------] 28/115                  Repaid and re-uploaded 115 chunks in 206.046789377s

I’m busy, have to go, maybe back later today, if not, then tomorrow.

[joshuef]

@Toivo

Uploading is very slow, has been about 15 minutes and a 60 MB file is still uploading. A lot of

chunks were not stored. Repaying them in batches.

Ahhh, yeh. I can see we’ve lost some nodes already. So mem is better, than our tests end of last week by a fair bit, but seems we won’t cope with general load like this. (Most nodes look ~115mb, some up to 250… and its those machines we’re losing nodes on).

[Josh]

@joshuef

So ram usage is expected to be higher.

Got it, I had packed the nodes in tight expecting problems to occur late in the game at just over 100MB usage but my machines were buckling so I restarted, this time with specifying logs and a lot less nodes.

But they are not immediately looking lively, very few connections.

[happybeing]

@Josh

But they are not immediately looking lively, very few connections.

Twenty nodes and no PUTS but also much less logfile activity than I’m used to.

My client is also taking an age to grab from the faucet so I’m going to restart it. I had it work a few minutes ago though.

Client seems hung at “Connecting to The SAFE Network…” for faucet and upload. I can get my wallet balance though.

@joshuef my nodes and client both seem not to be able to do anything useful. I did get the faucet to work a couple of times but no more on laptop or cloud instances, they just sit there ‘connecting’. My twenty nodes in the cloud are showing no activity and all still at 20MB-21MB RAM (no PUTS/GETS, some errors at start but now just resource updates).

[Vort]

State of my node:

• Uptime 2 hours;
• 3% CPU usage;
• 148 MB of RAM;
• 280 MB of logs;
• 0.8 MB/s send, 2 MB/s receive;
• 0 files in record_store.

~3 MB/s is too much resources consumed by “network noise” in my opinion.

[Toivo]

Node one RAM 672MB
Node two RAM 148MB

Big difference.

@Toivo

Uploading is very slow, has been about 15 minutes and a 60 MB file is still uploading.

Still uploading, been more than an hour.

@Vort

~3 MB/s is too much resources consumed by “network noise” in my opinion.

I see that too.

[Vort]

Downloads are not working (d435365368fe892be5bfb62f218fb35333d3b2815661c9eebac6d8c271186f52), network noise is at 5 MB/s now → I see no sense in further testing and I’m stopping my node.

[Josh]

@happybeing

Twenty nodes and no PUTS but also much less logfile activity than I’m used to.

I think this one has has hit the grave.

[JPL]

Halloween launch was always tempting fate.