DiskNet [May 16th Testnet 2023] **Now Offline**

:tada: Surprise! :tada: We’re launching a community testnet to test out the various tweaks and changes since JoshNet!

We have moved from storing the chunks in memory to storing them onto the disk. You can view the chunks at /node_root_dir/node_data/record_store.

We have other fixes which drastically reduces the memory usage by preventing concurrent dialling among peers and by replicating only when there is a churn in the close group. Also, we have revised our strategy for populating the node’s Routing Table, eliminating the need for random network look-ups.

There are 100 nodes running currently and each should have capacity of ~5.5 GB.

Note: please don’t try to join the network with a home node as it will screw things up. Cloud nodes only this time.

The goal of this testnet is to:

  • Ensure nodes are successfully connecting to peers, which can be validated by examining log activity.
  • Verify the disk-backed DataStore is working as intended.
  • Find potential causes of memory spikes, if any.

Running things

Network Address

To connect to the DiskNet, we should set the network address using the SAFE_PEERS env variable or use the --peer= argument. Use any of the following address to connect to the network,

export SAFE_PEERS=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB

# fall-back addresses
/ip4/3.9.177.176/udp/12000/quic-v1/p2p/12D3KooWBqsnvR5R4d7xn8ZPUX8skuYvXnJbsf6VPR729qEi9JZU
/ip4/13.40.28.114/udp/12000/quic-v1/p2p/12D3KooWBsWbQPhZYCJgCDn4iYqk8AArnmPrkzUBcEzDYkAjR3md
/ip4/13.40.72.208/udp/12000/quic-v1/p2p/12D3KooWFr9Df8o8tpj2WZvfUU4p3xsTteYdEsi26rwBmwBvjSDF

Using the Client

To put/get files you’ll need to use the safe client. Which you can grab for your platform from GitHub. Once you have the client, you need to either set the SAFE_PEERS environmental variable or use the --peers= argument with any of the above network addresses

Now to upload a directory/file to the network, use the following command:

# using the SAFE_PEERS variable
export SAFE_PEERS=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB
safe files upload -- <path>

# alternatively using the --peers argument. It should be set during each command
safe --peer=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB files upload -- <path>

The file-addresses of the content you’ve uploaded are saved locally and are used to enable automatic downloads. Use the following command to download them back to the ~/.safe/client/downloaded_files folder.

To download the content you’ve just uploaded:

$ safe files download

How to share files and download them?

When you upload you will also see, for example:

Successfully stored file "testnet.exe" to 332f45ae9482a8e03b5d5579f15fd9c8806fe4a2b30da2bc6e5ebb9a98fe4095

This wee hex string allows you to easily share files (without having to have the addresses locally).

safe files download -- testnet.exe  332f45ae9482a8e03b5d5579f15fd9c8806fe4a2b30da2bc6e5ebb9a98fe4095

(here you can change the target filename (i.e. testnet.exe) to whatever you want.)

Running a Node

The node (safenode binaries) connects to the Network similarly to the client, by passing the network address of any other public node. Use the SAFE_PEERS env variable or --peers arg as mentioned here to connect to the Network. It also needs a directory specified for the data it stores. And it’s a good idea to keep the logs in a directory.

$ safenode --root-dir=/tmp/safenode --log-dir=/tmp/safenode

Full NAT traversal is still a WIP and the node should be publicly visible for the other nodes to connect to it. Therefore it probably won’t work behind a home router yet.

So please don’t try to join with a home node as it will screw things up. Cloud nodes only for now.

36 Likes

So much for @Southside’s prediction :partying_face:

14 Likes

Yippeeeeeeeeeeeeeeeeeeeeeeeeeeeeee!!!
:heartpulse:

16 Likes

2 tests in a row, hat-trick coming up :cricket_bat_and_ball: :laughing:

12 Likes

It should read it will not work behind a home router. If you are on NAT please don’t connect a node, as it will just cause some trouble right now. The next test will clear that up, though.

EDIT: I should say, just in case, clients can connect from anywhere. No worries there.

24 Likes

I’m thinking of opening a Patreon Page where you can all pay me to make shite predictions, thus hastening the pace of development.

9 Likes

Any success in up/downloading, anyone?

It seems to get stuck when I try to put a small 36 kB jpg.

safe files upload -- topibott.jpg
Starting logging to directory: "/tmp/safe-client.log"
Instantiating a SAFE client...

Also, for other non-techies, copy-pasting this exact thing seems to wokr on Linux, but not on Windows:

export SAFE_PEERS=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB

Windows does not seem to recognize the export command.

5 Likes

Amended the OP :sunglasses:

8 Likes

Anyhow…

I would contend a “no nodes from behind NAT” is “limited functionality”

So there !!! – insert desperate grin emoji here —

Who will be first to break it by attempting a node from home?

7 Likes
4 Likes

The cock went up but the criminal remained flaccid

willie@gagarin:~/projects/maidsafe/safe_network$ safe files upload -- ~/cock01.jpg 
Removed old logs from directory: "/tmp/safe-client.log"
Starting logging to directory: "/tmp/safe-client.log"
Instantiating a SAFE client...
Storing file "cock01.jpg" of 146127 bytes..
Successfully stored file "cock01.jpg" to d724a3ff9b2746e042fa1e8db4c4e0ec5769097042911cc8a34197edc1342e47
Writing 58 bytes to "/home/willie/.safe/client/uploaded_files/file_names_2023-05-16_15-34-57"
willie@gagarin:~/projects/maidsafe/safe_network$ locate bigdunc
/home/willie/bigdunc.mp4
/home/willie/Videos/bigdunc.mp4
willie@gagarin:~/projects/maidsafe/safe_network$ safe files upload -- ~/bigdunc.mp4 
Removed old logs from directory: "/tmp/safe-client.log"
Starting logging to directory: "/tmp/safe-client.log"
Instantiating a SAFE client...
Storing file "bigdunc.mp4" of 30772064 bytes..
Did not store file "bigdunc.mp4" to all nodes in the close group! Network Error Outbound Error.
Writing 8 bytes to "/home/willie/.safe/client/uploaded_files/file_names_2023-05-16_15-36-00"

d724a3ff9b2746e042fa1e8db4c4e0ec5769097042911cc8a34197edc1342e47 for those who want Chicken Tonight

4 Likes

Wow! So fast next testnet?

If I manage to join DiskNet, I will be telling my grandchildren about it! :blush:

10 Likes

I’m not seeing nearly the activity I did last testnet. Does the ‘Connection closed’ event mean I am no longer a node, or is there just no traffic?

[2023-05-16T14:41:05.215392Z INFO safenode::peers_acquisition] Using contact peers from $SAFE_PEERS env var to intitiate contact with the network: ["/ip4/52.56.104.157/udp/12000/quic-v1/p2
p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB"]
[2023-05-16T14:41:05.215479Z INFO safenode]
Running safenode v0.1.0
=======================
[2023-05-16T14:41:05.216293Z INFO safenode] Starting node ...
[2023-05-16T14:41:05.216507Z INFO safenode::network] Node (PID: 260520) with PeerId: 12D3KooWHZjPfE7oSwxH6zVaCAYQGxJM6oiUMRhz1yD3D2iMf9Cj
[2023-05-16T14:41:05.220137Z INFO safenode::network::event] Local node is listening on "/ip4/127.0.0.1/udp/37831/quic-v1/p2p/12D3KooWHZjPfE7oSwxH6zVaCAYQGxJM6oiUMRhz1yD3D2iMf9Cj"
[2023-05-16T14:41:05.220254Z INFO safenode::network::event] Local node is listening on "/ip4/192.3.62.100/udp/37831/quic-v1/p2p/12D3KooWHZjPfE7oSwxH6zVaCAYQGxJM6oiUMRhz1yD3D2iMf9Cj"
[2023-05-16T14:41:05.220969Z ERROR safenode::node::api] Failed to dial 12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB: AlreadyDialingPeer(PeerId("12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvS
dJd3bYdA29q3H7GB"))
[2023-05-16T14:41:05.312301Z INFO safenode::network::event] Connected with PeerId("12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB")
[2023-05-16T14:41:05.900982Z INFO safenode::network::event] IdentifyEvent: Received { peer_id: PeerId("12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB"), info: Info { public_key: Ed25
519(PublicKey(compressed): 6a183364a2bbd2e0bd4dbb272dd8c0d1a38c305b6d983319b7cdc5fdca1668), protocol_version: "safe/0.1.0", agent_version: "safe/node/0.1.0", listen_addrs: ["/ip4/10.0.0.11
6/udp/12000/quic-v1", "/ip4/127.0.0.1/udp/12000/quic-v1", "/ip4/10.0.0.116/udp/12000/quic-v1", "/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7G
B", "/ip4/52.56.104.157/udp/12000/quic-v1", "/ip4/10.0.0.116/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB"], protocols: ["/safe/1", "/ipfs/kad/1.0.0", "/ipfs/
id/1.0.0", "/ipfs/id/push/1.0.0"], observed_addr: "/ip4/192.3.62.100/udp/37831/quic-v1" } }
[2023-05-16T14:41:05.901066Z INFO safenode::network::event] Adding peer to routing table, based on received identify info from PeerId("12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB"
): Info { public_key: Ed25519(PublicKey(compressed): 6a183364a2bbd2e0bd4dbb272dd8c0d1a38c305b6d983319b7cdc5fdca1668), protocol_version: "safe/0.1.0", agent_version: "safe/node/0.1.0", list
en_addrs: ["/ip4/10.0.0.116/udp/12000/quic-v1", "/ip4/127.0.0.1/udp/12000/quic-v1", "/ip4/10.0.0.116/udp/12000/quic-v1", "/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkD
MafTkpteBvSdJd3bYdA29q3H7GB", "/ip4/52.56.104.157/udp/12000/quic-v1", "/ip4/10.0.0.116/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB"], protocols: ["/safe/1",
"/ipfs/kad/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0"], observed_addr: "/ip4/192.3.62.100/udp/37831/quic-v1" }
[2023-05-16T14:41:05.901313Z DEBUG safenode::node::api] PeerAdded: 12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB
[2023-05-16T14:41:05.901340Z INFO safenode] Connected to the Network
[2023-05-16T14:41:05.989222Z INFO safenode::network::event] IdentifyEvent: Sent { peer_id: PeerId("12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB") }
[2023-05-16T14:41:25.312970Z INFO safenode::network::event] Connection closed to Peer 12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB - Dialer { address: "/ip4/52.56.104.157/udp/12000
/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB", role_override: Dialer } - Some(KeepAliveTimeout)
2 Likes

Managed to get the cock down, but I cannot get anything up!

1 Like

Hey @Toivo ! Looks like the client was not able to connect to the provided SAFE_PEERS. Could you try again by setting the SAFE_PEERS to any of the fall-back addresses provided above?

Could you try with SET SAFE_PEERS=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB on windows?

5 Likes

You can get pills for that nowadays. Check your spam folder.

2 Likes

A mixed bag of attempted uploads…

Storing file "cfm-idleRear.wav" of 326360 bytes..
Successfully stored file "cfm-idleRear.wav" to ca385d9deb2ca277da29091718735d171c81e526d4bd85e81c5b2d1477dff113
Storing file "cfm-buzz.wav" of 127386 bytes..
Successfully stored file "cfm-buzz.wav" to 800d5dba74bd1ad5d081ef770ae150e0c3cf7b7399af95dfa5c2bd315ad7e4b8
Storing file "cfm-windrush.wav" of 61374 bytes..
Successfully stored file "cfm-windrush.wav" to 3db4e3561784047636d213c6304ea547739b6c5ebcebefcfac5ca680e9af5fe1
Storing file "cfm-idleClose.wav" of 1044524 bytes..
Successfully stored file "cfm-idleClose.wav" to 41e09ad191310048c61aa740b53d6be17659cc53c3c1606717f3109aab42a587
Storing file "cfm-idleDistSide.wav" of 658550 bytes..
Successfully stored file "cfm-idleDistSide.wav" to 51a6aaccd039ffbf6aabb8af8c185b62a45f9396a560ca5a8dddfdfabc369ca5
Storing file "cfm-startup.wav" of 2658348 bytes..
Did not store file "cfm-startup.wav" to all nodes in the close group! Network Error Outbound Error.
Storing file "cfm-spoolCloseRear.wav" of 2533452 bytes..
Did not store file "cfm-spoolCloseRear.wav" to all nodes in the close group! Network Error Outbound Error.
Storing file "cfm-buzzDist.wav" of 184024 bytes..
Successfully stored file "cfm-buzzDist.wav" to d51c860be865541634361d64afefa9678121c8a7f4b2eca7c997eeed2f46ed84
Storing file "cfm-spool.wav" of 379904 bytes..
Successfully stored file "cfm-spool.wav" to 9efdfdc6d533f0843177b71516fd9e55fd1a5507f8ad2a65e2907e64635d10d0
Storing file "cfm-rumble.wav" of 354204 bytes..
Successfully stored file "cfm-rumble.wav" to dbb86e1916969bd56a0d4e955b77154fd2ae74e26d8b1d1c57e18224cb464a27
Storing file "cfm-rumbleSide.wav" of 353014 bytes..
Successfully stored file "cfm-rumbleSide.wav" to 61e8c29b3bbe4253c1c108c2157d17b2b8ebb85e2a93d852c88147b7f5f70a69
Storing file "cfm-idle.wav" of 321594 bytes..
Successfully stored file "cfm-idle.wav" to eace216b76fc36621b0d2e737c755a1837247ebd9d0e7ee2571b18a39fc787e8
Storing file "cfm-starter.wav" of 2478124 bytes..
Did not store file "cfm-starter.wav" to all nodes in the close group! Network Error Outbound Error.
Storing file "cfm-shutdown.wav" of 4006956 bytes..
Did not store file "cfm-shutdown.wav" to all nodes in the close group! Network Error Outbound Error.
Storing file "cfm-reverse.wav" of 317654 bytes..
Successfully stored file "cfm-reverse.wav" to 968254f70a61d1a4be7aff6930f0485cf8a2394eb465a23f3f2d60092243f89c
Storing file "cfm-lowspool.wav" of 298472 bytes..
Successfully stored file "cfm-lowspool.wav" to 9581a57b975f6ea1478592323cf99446fe739c7219aaa29f3ec4e1420087dfc2
Storing file "cfm-hum.wav" of 741010 bytes..
Successfully stored file "cfm-hum.wav" to 7e6cee75dc7f021e36af41f89a411406587b765135dd0c7d7e1ad3e4eb24b324
Storing file "320_start2_2.wav" of 2452800 bytes..
Did not store file "320_start2_2.wav" to all nodes in the close group! Network Error Outbound Error.
Storing file "cfm-buzz.wav" of 306788 bytes..
Successfully stored file "cfm-buzz.wav" to 5871619e6208da54f4790f21775729f61aad0c2363ade1b4e4b70a8cc7c85b6f
4 Likes

None of them work on my Ubuntu machine.

Did that, and managed to download something and upload about 4MB jpg. Both very quickly.

Will be AFK for a while, worst case for the rest of the night. :sleepy:

Are you behind a home router?

Looking at the logs the ConnectionClosed event was triggered because the connection to the bootstrap peer was inactive for a while, thus it got timed out. The peer is still maintained in the routing table; it’s just that the direct connection to it has been closed.

I’m not seeing nearly the activity I did last testnet.

If you are not behind a NAT, could you kindly try connecting to multiple bootstrap peers instead of just one. The $SAFE_PEERS takes in comma separated values, so we can do something like this:

export SAFE_PEERS=/ip4/52.56.104.157/udp/12000/quic-v1/p2p/12D3KooWGxWnfChkZTLTqFCkDMafTkpteBvSdJd3bYdA29q3H7GB,/ip4/3.9.177.176/udp/12000/quic-v1/p2p/12D3KooWBqsnvR5R4d7xn8ZPUX8skuYvXnJbsf6VPR729qEi9JZU,/ip4/13.40.28.114/udp/12000/quic-v1/p2p/12D3KooWBsWbQPhZYCJgCDn4iYqk8AArnmPrkzUBcEzDYkAjR3md,/ip4/13.40.72.208/udp/12000/quic-v1/p2p/12D3KooWFr9Df8o8tpj2WZvfUU4p3xsTteYdEsi26rwBmwBvjSDF
6 Likes