NoEncryptionNet [07/02/24 Testnet] [Offline]

After QuicNet blazed itself to the ground, we’re back with a network that

  1. Does not encrypt data at nodes. (In order to see how much that contributed to mem-spikes we saw).
  2. Derives payment keys from the PeerId key, so restarted nodes should be able to accept payments now.
  3. Has more logs

Here we’re looking to see

  • How this lack of encryption impacts performance
  • Can we upgrade this network (in theory we can, but we’ll see how that goes when we’ve new versions out…)
  • Are memory leaks still afoot? What can we see with our increased logging here.

For those running nodes: please enable a good amount of log files to be kept (~50), this will take up more space, but we’re not anticipating a long testnet here :+1:

Network Details

Node version: 0.103.45
Client version: 0.89.49
Faucet url: 178.128.166.148:8000

We have 101 droplets running a total of 1001 nodes. One droplet has 2vcpu and 4GB of memory and 10 nodes (down from 40 to allow for more detailed logging this run!).


If you are a regular user, see the ‘quickstart’ section for getting up and running.

If you are a first-time user, or would like more information, see the ‘further information’ section.


Quickstart

If you already have safeup, you can obtain the client and node binaries:

safeup client --version 0.89.49
safeup node --version 0.103.45

Run a Node

Linux/macOS:

safenode

Windows:

safenode

Check local node’s reward balance

Your local node’s peer id will be printed to the terminal on startup with an example command). (You can also retrieve this from the node directory.)

safe wallet balance --peer-id="<local-node-peer-id>"

Connect to the Network as a Client

Linux/macOS:

safe wallet get-faucet 178.128.166.148:8000
safe files upload <directory-path>

Windows:

safe wallet get-faucet 178.128.166.148:8000
safe files upload <directory-path>

To do this with non-default batch-sizes (along with SAFE_PEERS set as above):

safe files upload --batch-size 40 <directory-path> 

40 being the integer value you want to set


Further Information

You can participate in the testnet either by connecting as a client or running your own node.

Connecting as a client requires the safe client binary; running a node requires the safenode binary.

Obtaining Binaries

We have a tool named safeup which is intended to make it easy to obtain the client, node, and other utility binaries.

Installing Safeup

On Linux/macOS, run the following command in your terminal:

curl -sSL https://raw.githubusercontent.com/maidsafe/safeup/main/install.sh | bash

On Windows, run the following command in a Powershell session (be careful to use Powershell, not cmd.exe):

iex (Invoke-RestMethod -Uri "https://raw.githubusercontent.com/maidsafe/safeup/main/install.ps1")

On either platform, you may need to restart your shell session for safeup to become available.

Installing Binaries

After obtaining safeup, you can install binaries like so:

safeup client # get the latest version of the client
safeup client --version 0.89.49 # get a specific version

safeup node # get the latest version of the node
safeup node --version 0.103.45 # get a specific version

safeup update # update all installed components to latest versions

When participating in our testnets, it is recommended to use a specific version. In our project we release a new version of the binaries every time we merge new code. This happens frequently, so over the lifetime of a testnet, many new releases will probably occur. So for participating in this particular testnet, you may not want the latest version.

The binaries are installed to ~/.local/bin on Linux and macOS, and on Windows they go to C:\Users\<username>\safe. Windows doesn’t really have a standard location for binaries that doesn’t require elevated privileges.

The safeup tool will modify the PATH variable on Linux/macOS, or the user Path variable on Windows. The effect of this is that the installed binaries will then become available in your shell without having to refer to them with their full paths. However, if you’re installing for the first time, you may need to start a new shell session for this change to be picked up.

Running a Node

You can participate in the testnet by running your own node. At the moment, you may not be successful if you’re running the node from your home machine. This is a situation we are working on. If you run from a cloud provider like Digital Ocean or AWS, you should be able to participate.

You can run the node process like so:

# Linux/macOS
safenode

# Windows
safenode

This will output all the logs to the filesystem, with the location of logs being platform specific:

# Linux
~/.local/share/safe/node/<peer id>/logs

# macOS
/Users/<username>/Library/Application Support/safe/node/<peer id>/logs

# Windows
C:\Users\<username>\AppData\Roaming\safe\node\<peer-id>\logs

If you wish, you can also provide your own path:

# Linux/macOS
SN_LOG=all safenode --log-output-dest <path>

# Windows
$env:SN_LOG = "all"; safenode --log-output-dest <path>

The advantage of using the predefined data-dir location is you can run multiple nodes on one machine without having to specify your own unique path for each node and manage that overhead yourself.

Connecting as a Client

You can use the safe client binary to connect as a client and upload or download files to/from the network.

Using the Client

You’ll first need to get some Safe Network Tokens:

safe wallet get-faucet 178.128.166.148:8000

You can now proceed to use the client, by, e.g., uploading files:

safe files upload <directory-path>

To download that same content:

safe files download

This will download the files to the default location, which is platform specific:

# Linux
~/.local/share/safe/client/downloaded_files

# macOS
/Users/<username>/Library/Application Support/safe/client/downloaded_files

# Windows
C:\Users\<username>\AppData\Roaming\safe\client\downloaded_files

To download to a particular file or directory:

safe file download [directory/filename] [NetworkAddress]

Troubleshooting

Cleanup

If you’ve used previous versions of the network before and you find problems when running commands, you may want to consider clearing out previous data (worthless DBCs from previous runs, old logs, old keys, etc.).

# Linux
rm -rf ~/.local/share/safe

# macOS
rm -rf ~/Library/Application\ Support/safe

# Windows
rmdir /s C:\Users\<username>\AppData\Roaming\safe

If you encounter a problem running any of our binaries on Windows, it’s possible you need the Visual C++ Redistributable installed.

33 Likes

Congratulations on this amazing testnet! :tada:

21 Likes

Ten cloud nodes started

Up and running, but notice that my pristine nodes are recording GETS without records. Might be the ordering of messages or a miscounted message by vdash but just noting for now:

21 Likes

Let’s see how UDP based network will behave with my overloaded connection :slight_smile:

15 Likes

No Encryption hey. Well the title tells half the truth. Clients still encrypt in case anyone wonders.

Will someone get an unencrypted chunk stored on their node, will anyone be looking for it???

These questions and more will possibly be answered in this episode of “The Year of Solutions testnets”TM

16 Likes

Blazing fast again. First 50M file uploaded no problem. Second one got stuck at 70 chunks, with a recurring error of not enough peers in k-bucket. Waited a few mins and retried, and the remaining 31 chunks uploaded no problem - again, blazing fast.

21 Likes

I made two uploads: 62 MiB and 110 MiB.
Both started with fast enough speed.
First one also finished fast.
Second one had problems: lags at the end and several retries.
But at the end both files were uploaded (and downloaded afterwards) succesfully.

16 Likes

In with 73, 50 logs kept as requested. A quick first gander at mem right after start up is that it looks more like previous tests. I started higher with quicnet, time will tell.

16 Likes

4 machines up, altogether 160 nodes, so far it is less resource hungry, we will se more when people start hammering it.

EDIT: 260 nodes, I am bad at math :smiley:

13 Likes

and we’re offfff to the races!! :laughing: :partying_face:

Will see how much I can upload before having to hit the hay.

11 Likes

well … i didn’t want to stress my internet connection and/or the rpi4 too much and just started 3 nodes …

already “earned” 115 n€ worth of worthless test-coin!!!
awesome dashboard @happybeing and thanks @aatonnomicc for the script

pi not looking very busy to be honest …

…but ofc i made a mistake and just recognized i didn’t respect the 50 log-file request by maidsafe …

11 Likes

I made an experiment:

I made 5 versions of one photograph. Version 0 is a landscape without any alterations as an uncompressed -tiff format, about 69MB. Versions 1 - 4 I edited one pixel in every corner of the image, to see how editing affects to how it’s chunks are deduplicated.

Results:

Version 0: Among 139 chunks, found 0 (of course)
Version 1: Among 139 chunks, found 0
Version 2: Among 139 chunks, found 135
Version 3: Among 139 chunks, found 134
Version 4: Among 139 chunks, found 134

Uploading was superfast otherwise, but the last few chunks took a while in every version.

I may try other versions later, with different edits.

The original blank image you can download from here:

safe files download "Landscape_0.tif" fc4b349dd4dce1f07ed31724dda85c819206f3530ab805284204839397dda4bd
17 Likes

Super fast, and nodes memory consumption under control so far.
:muscle:

11 Likes

sorry for the churn - stopped my nodes and started 5 with 50 log files :face_with_monocle:

earning fast!

more than 1µ€ since start of those 5 nodes!

9 Likes

It would be interesting to have one or few large files uploaded in the beginning of the testnet, made available in OP. We could test downloading them to see how they survive after someone reports a number of nodes gone offline.

Or if anyone wants to upload now, great! I can’t up- or download those, as it would crash our home LAN :face_with_diagonal_mouth:

Connected to this discussion:

2 Likes

I’m currently uploading mixtral-8x7b-instruct-v0.1.Q6_K.gguf (35.7gb).

I suspect I’m not the first due to the rapid speed (about 2x previous test, which was about 10x the one before that). But, the additional speed could also be due to no encryption, or also that I selected batch size 80 (not sure what the default is?).

Edit: I was the first. The network is just blazing fast!

6 Likes

Now i see why my internet connection is suddenly maxed out :smiley:

9 Likes

I’m uploading a largish file now 2GB or so … (large for me anyway). It does appear to be going faster than in the last testnet, but early still. Will see where it is in an hour or so.

5 Likes

Can anyone explain what these messages mean?
Replication Fetcher doesn't have free capacity. Currently has 841 entries in queue.

At first I thought it means that network is full.
But then I looked at first logs and I see the same messages right after node was started.

1 Like