QuicNet [30/01/24Testnet] [Offline]

NewYearNet was probably our longest testnet yet. Although we started hitting the limits of diskspace as our log levels were still quite high (and no log compression was set), which necesitated us killing it off.

So in this testnet, we’ll be logging less and archiving those logs more frequently on our nodes. Which could well keep us going longer (barring any need to upgrade the nodes themselves).

The major change here though, is the switch from tcp to quic as the default communication layer. This means that AutoNat wont be working for us, so if your node does not manage to connect, the automatic exit on the node side of things will not be working.

We’re looking to assess the quic performance, both in terms of files up/down, but also in terms of resource usage of nodes. So please keep an eye on any changes in performance or utility you’re perceiving here compared to the last network :bowing_man:

The only other change of note here is that nodes will now be encrypting and decrypting records before they’re written to disk (regardless of what the client is doing). So now no data at all should be exposed to normal node operators.

Network Details

Node version: 0.103.25
Client version: 0.89.29
Faucet url: 157.245.40.226:8000

We have 51 droplets running a total of 2001 nodes. One droplet has 2vcpu and 4GB of memory.


If you are a regular user, see the ‘quickstart’ section for getting up and running.

If you are a first-time user, or would like more information, see the ‘further information’ section.


Quickstart

If you already have safeup, you can obtain the client and node binaries:

safeup client --version 0.89.29
safeup node --version 0.103.25

Run a Node

Linux/macOS:

safenode

Windows:

safenode

Check local node’s reward balance

Your local node’s peer id will be printed to the terminal on startup with an example command). (You can also retrieve this from the node directory.)

safe wallet balance --peer-id="<local-node-peer-id>"

Connect to the Network as a Client

Linux/macOS:

safe wallet get-faucet 157.245.40.226:8000
safe files upload <directory-path>

Windows:

safe wallet get-faucet 157.245.40.226:8000
safe files upload <directory-path>

To do this with non-default batch-sizes (along with SAFE_PEERS set as above):

safe files upload --batch-size 40 <directory-path> 

40 being the integer value you want to set


Further Information

You can participate in the testnet either by connecting as a client or running your own node.

Connecting as a client requires the safe client binary; running a node requires the safenode binary.

Obtaining Binaries

We have a tool named safeup which is intended to make it easy to obtain the client, node, and other utility binaries.

Installing Safeup

On Linux/macOS, run the following command in your terminal:

curl -sSL https://raw.githubusercontent.com/maidsafe/safeup/main/install.sh | bash

On Windows, run the following command in a Powershell session (be careful to use Powershell, not cmd.exe):

iex (Invoke-RestMethod -Uri "https://raw.githubusercontent.com/maidsafe/safeup/main/install.ps1")

On either platform, you may need to restart your shell session for safeup to become available.

Installing Binaries

After obtaining safeup, you can install binaries like so:

safeup client # get the latest version of the client
safeup client --version 0.89.29 # get a specific version

safeup node # get the latest version of the node
safeup node --version 0.103.25 # get a specific version

safeup update # update all installed components to latest versions

When participating in our testnets, it is recommended to use a specific version. In our project we release a new version of the binaries every time we merge new code. This happens frequently, so over the lifetime of a testnet, many new releases will probably occur. So for participating in this particular testnet, you may not want the latest version.

The binaries are installed to ~/.local/bin on Linux and macOS, and on Windows they go to C:\Users\<username>\safe. Windows doesn’t really have a standard location for binaries that doesn’t require elevated privileges.

The safeup tool will modify the PATH variable on Linux/macOS, or the user Path variable on Windows. The effect of this is that the installed binaries will then become available in your shell without having to refer to them with their full paths. However, if you’re installing for the first time, you may need to start a new shell session for this change to be picked up.

Running a Node

You can participate in the testnet by running your own node. At the moment, you may not be successful if you’re running the node from your home machine. This is a situation we are working on. If you run from a cloud provider like Digital Ocean or AWS, you should be able to participate.

You can run the node process like so:

# Linux/macOS
safenode

# Windows
safenode

This will output all the logs to the filesystem, with the location of logs being platform specific:

# Linux
~/.local/share/safe/node/<peer id>/logs

# macOS
/Users/<username>/Library/Application Support/safe/node/<peer id>/logs

# Windows
C:\Users\<username>\AppData\Roaming\safe\node\<peer-id>\logs

If you wish, you can also provide your own path:

# Linux/macOS
SN_LOG=all safenode --log-output-dest <path>

# Windows
$env:SN_LOG = "all"; safenode --log-output-dest <path>

The advantage of using the predefined data-dir location is you can run multiple nodes on one machine without having to specify your own unique path for each node and manage that overhead yourself.

Connecting as a Client

You can use the safe client binary to connect as a client and upload or download files to/from the network.

Using the Client

You’ll first need to get some Safe Network Tokens:

safe wallet get-faucet 157.245.40.226:8000

You can now proceed to use the client, by, e.g., uploading files:

safe files upload <directory-path>

To download that same content:

safe files download

This will download the files to the default location, which is platform specific:

# Linux
~/.local/share/safe/client/downloaded_files

# macOS
/Users/<username>/Library/Application Support/safe/client/downloaded_files

# Windows
C:\Users\<username>\AppData\Roaming\safe\client\downloaded_files

To download to a particular file or directory:

safe file download [directory/filename] [NetworkAddress]

Troubleshooting

Cleanup

If you’ve used previous versions of the network before and you find problems when running commands, you may want to consider clearing out previous data (worthless DBCs from previous runs, old logs, old keys, etc.).

# Linux
rm -rf ~/.local/share/safe

# macOS
rm -rf ~/Library/Application\ Support/safe

# Windows
rmdir /s C:\Users\<username>\AppData\Roaming\safe

If you encounter a problem running any of our binaries on Windows, it’s possible you need the Visual C++ Redistributable installed.

34 Likes

And for vdash fans the latest update is only a few minutes old, and let’s you use live fiat prices in your currency of choice:

25 Likes

Early signs very good. Faucet was blinding fast and even my uploads over mobile broadband are going well, really fast compared to previous (1 chunk/sec so far).

Here are my cloud nodes:

Update - cash rolling in after 1 hour:

After 90 minutes earnings range is 15:1 (highest 3835 nanos to lowest 247 nanos) so I’m starting nodes 11 to 13 to see if any can catch and beat node 6. Node 11 began earning after 5 minutes (which is when I started 12 & 13). Node 12 began earning within a minute and 13 withing 2 mins. Node 11 hasn’t earned any more so the new nodes are gaining very early on!

NOTE: In about 2 minutes node 12 has reach 1/3 of the total of Node 6 which has been running for 90 minutes. It will be interesting to see if this continues when node 12’s Store Cost (50) passes Node 6’s Store Cost (72) as Node 12 passes 50% of Node 6’s earnings. This is quite a race, 12 is holding with 11 and 13 moving up but Node 6 :horse_racing: is stretching its lead again because of its higher storage cost!

Breaking: Node 13 (433 nanos) and Node 12 (339 nanos) after 20 mins have overtaken Node 6 (329 nanos in 113 minutes)!

After 37 minutes node 13 is in the top 8, ahead of three nodes which have been runnning for 131 minutes, and poor Node 6 is at the bottom again

Node 6 Exploded about 4 hours ago! Possible bug? cc @qi_ma

Here’s the table of all nodes (see also above, that Node 6 was the laggard):

Node 6 was killed after its RAM use rocketed. Here are the graphs which show RAM use started to rise rapidly a few hours ago, while the other graphs look reasonable. GETS were relatively high in that period but the total GETS are less than many other nodes. 1 hour columns:


Logs for Node 6:
safenode6.zip (1.1 MB)

FINAL STATE (after testnet offline)

27 Likes

Yay! A new testnet. I hope everything goes well.

8e3j1q

14 Likes

Also encouraging, for the first time I can run nodes over the mobile broadband connection, although it looks like they are having difficulty. The log says “peers found_and_sorted” but I’m not seeing any healthy stats. No PUTS, GETS Peers etc, just errors and RAM use reported.

Enclosing a log in case it explains the above… safenode.zip (14.0 KB)

Update: I’m going to stop these mobile broadband at home nodes as they are not doing anything useful. Still running but all the above zero’s are still there. I notice that all the errors occurred during start up, nothing in the last hour except the resource log messages confirming constant RAM use.

19 Likes

Awesome to see a new testnet!

Currently uploading at over 300 chunks per minute… let’s see how it holds up, but blazing fast compared to before thus far :smiley:

20 Likes

Here we go! :smiley:
I’m curious to see the progress, as soon as I get my act together I’ll join the tests.

17 Likes

Pluck me that’s fast as duck

13 Likes

On the client side over mobile broadband things went far, far better than any recent testnet. Previously I was able to upload only a handful of small files. Now I’m getting ~100 chunks / minute and was able to upload a decent sized directory tree (a 29MB node project of hundreds of files) in about 1 hour:

Among 6026 chunks, found 280 already existed in network, uploaded the leftover 5746 chunks in 61 minutes 46 seconds
**************************************
*          Payment Details           *
**************************************
Made payment of 0.000267092 for 5746 chunks
Made payment of 0.000043759 for royalties fees
New wallet balance: 699.999681527

real	61m48.873s
user	9m37.410s
sys	3m4.111s

I’m a little skeptical about 280 chunks already being on the network, but if other’s have been uploading node projects it is plausible.

19 Likes

This is very encouraging as QUIC is for sure the way we need to go and it seems, at least so far, that it’s quite stable and efficient. TCP is not a good p2p protocol, so this is great news.

26 Likes

Well, I’ve uploaded about 8gb of a big file (37gb) in under an hour. For me, this is about 10x faster to upload than previous test nets. I never made it past about 3-4gb before.

Huge improvement it seems!

Also, first test net where I’ve been able to run a node from home.

I know it’s early days, but I’m excited about this apparent progress… as long as MaidSafe aren’t faking it somehow :laughing:

22 Likes

Yes its all very pheasant.
Sadly, I will miss much of this testnet cos I am in foreign parts and dont have my Hetzner login with me.

14 Likes

Oh mister, how could you :smiley: :smiley:

No it’s just QUIC to blame here

14 Likes

Be interested in the download here?

10 Likes

Well, QUIC is certainly living up to its name. Brilliant step forward in terms of speed here… let’s see how the stability holds up!

Edit: Update on my upload - over an hour in and still uploading at hundreds of chunks per minute. I think this works out at an upload rate of about 2.2mb per second so far. Not bad!

20 Likes

Is port forwarding needed at all now in any situation

3 Likes

It downloaded without error in 32 minutes, but the directory sizes are wildly different:
29M pr
18M Downloads/safe_files

I can’t easily do a proper compare because the upload was a deep directory tree, but it looks too different to be ok, even if skipping lots of small files.

5 Likes

I have set up port forwarding, but it still didn’t work previously.

I will try again without port forwarding at some point to see if that fails.

While I have a node connected, and storing chunks, it hasn’t received payments yet, so perhaps there’s some issue due to my from-home setup… we’ll see how it develops.

8 Likes

Yes, there is no NAT traversal here. That is still ongoing in libp2p land

12 Likes