Fleming Testnet v2 is here!
v2 Changelog
Fleming testnet v1 was released only a few days ago on April 8th, so this is the first of what we expect will be several testnet iterations which fix bugs, improve the user experience, plus tweaks and polishes the features the testnet contains.
The fixes and improvements for v2 are:
1. Automatic join retry
To make the node joining process smoother, we have implemented a simple loop which automatically retries joining your node to the testnet every 3 minutes until it is successful. You no longer need to manually re-run the $ safe node join command, or write and run your own script to keep retrying.
Note that we are in the process of implementing a more refined node join solution where your node would join a queue of infants, and when a space becomes available in a section, whichever infant in the queue is closest to that section’s XorName space will be selected to be promoted to join that section as an adult. This didn’t quite make it in to the v2 cut today, but should be added in an upcoming iteration.
2. Allow more nodes to join
We have also removed some limits we had on new nodes joining the testnet; in v1 we would wait until a certain percentage of space was filled before allowing a new node to join, but now we will also allow a new node to join as soon as any current node becomes full, or on a current node leaving. This should allow more nodes onto the testnet.
3. Fixed dead Elders
In testnet v1 we observed a particular section (prefix 10) become stuck in a DKG loop - it seems this was due to two community nodes in that section being promoted to be Elders, but it turned out that other nodes could not establish a direct connection to them to perform Elder duties. This led to a (successful) DKG vote to demote those Elders to Adults, then a further DKG vote to promote the two oldest Adults to be Elders…which, you guessed it, were the same two nodes that couldn’t be connected to previously. This loop continued endlessly and the section did not perform any other actions in the meantime, causing some of the issues reported in the testnet v1 topic and others.
The simple fix for this was to only allow a node to be promoted to Elder if we can establish a connection to it, otherwise it will remain as an Adult regardless of its node age.
Note that remaining as an Adult is not a disadvantage to this node, it will continue earning rewards in that role. This way we are not excluding anybody from the network just because they are behind problematic NATs, etc.
What happens to testnet v1?
Fleming testnet v1 has now been taken offline, with all testnet v1 data deleted.
This is to allow us to concentrate on any issues that arise from the new testnet v2, and avoid any community confusion over which testnet they are connected to, or any errors as a result of contamination from v1 configurations and data. We will be asking that everyone who attempts to use testnet v2 removes their $HOME/.safe folder before trying to interact with testnet v2, i.e. $ rm -rf $HOME/.safe/ to ensure there is no contamination.
All of the bootstrap nodes for v1 that we had on Digital Ocean have been destroyed, with new, clean D.O. nodes created for testnet v2 - please follow the full list of instructions below to remove artifacts of v1 from your machine and to get you started with testnet v2.
Are We Working on Other Fixes?
Yes. Multiple other fixes and improvements are already in progress, some even from before we released testnet v1. One of the main priorities we have at the moment is completing the implementation of Anti Entropy across the board - we believe that this will resolve many of the errors that people have been seeing in their logs, and make the testnets even more reliable and scalable. This, along with others, are in progress for future testnet iterations.
We also anticipate that the feedback from this testnet iteration will direct us where else to aim resources as a priority.
Where can I report any issues found?
If you come across any issues in your testing, start by checking the Known Issues section of this post (see below) to see if it has already been acknowledged . If it has not yet been added, we would like you to report it in the Online Spreadsheet for Testnet Results and Issues v2 - see the section below.
We will monitor and investigate reported issues as soon as we can.
Online Spreadsheet for Testnet Results and Issues v2
If you will be uploading data to the testnet and/or providing safes (nodes), please consider posting your data and results at:
SN Testnet Review v2
(Massive thank you to @VaCrunch for creating this spreadsheet 
)
This is a 2nd version of this spreadsheet, with the version used for testnet v1 now locked.
Even if you don’t post your data and results, you can record any error messages and related information in the Error_Msgs sheet to help the development team keep these organised.
Ideally we can keep as much data, results and errors in this one location to allow us to more easily analyse it.
For those posting:
• No need to Sign In. The white cells are for your data entry.
• One row for each of your devices.
• Scroll down until you get to the first empty row.
• You do not need to use your Forum name for your ID, but please use the same ID for all your devices.
• Supporting/analysis tabs are at bottom of screen: Error_Msgs, Summary, Thumbnails, Charts, Matrix, Top 10, Map, Resources, Lists, and Comments.
• Use the View/Zoom feature at top to reduce the visible size of the spreadsheet to match your screen. 75% will be the best fit for most people.
• If you choose not to record the name of your country please select “Unspecified” at the bottom of the list.
• If you change the number of nodes (safes) you are running, just edit the existing figure, do not add another line for the same device.
Known Issues
Any reproducible issues that we, or the community, come across will be added to this section:
- We are seeing reports from testnet v1 of performance issues, particularly around
$ safe files put ....operations with larger files/folders. There has been no optimisation of write times, nor any general analysis of speed so far, with all efforts concentrated on getting features in place and fully functional before we look at it from a performance perspective.
Let’s try it out!
Please read and follow the instructions below carefully.
Installing/Updating to Latest
We must kill any running sn_node or sn_authd processes on your machine, the simplest way is via your terminal window:
Windows users:
$ taskkill /IM "sn_node" /F
$ taskkill /IM "sn_authd" /F
macOS and Linux users:
$ pkill sn_node
$ pkill sn_authd
To avoid any pollution from testnet v1 and its data, we ask that everyone removes their $HOME/.safe/ directory, then installs the CLI, authd and node again.
$ rm -rf $HOME/.safe
Note that before moving onto the steps below, Windows users may be required to install specific software before being able to install the CLI - see full detailed instructions for Windows here.
Now download and install the verified latest CLI binary (v0.23.1) via our install script with the following command:
$ curl -so- https://sn-api.s3.amazonaws.com/install.sh | bash
This script downloads the correct CLI binary for your OS (Windows, macOS or Linux), installs it in the correct directory, creating that directory if it doesn’t exist, and adds it to your system PATH.
You may need to restart your terminal window at this point for any changes to your system PATH to take effect. You can now confirm whether the CLI is installed and set up correctly:
$ safe -V
sn_cli 0.23.1
Next, you should install the Authenticator daemon and Node with the two commands below:
$ safe auth install
$ safe node install
You should now be equipped with the latest CLI, authd and node.
Note that you can find full installation instructions in our user guide:
Joining the Testnet
As with previous public testnets, we are hosting some elders and adults on Digital Ocean to kick off the Network. These act as hardcoded contacts which bootstrap you to the network, therefore you will need a network configuration file to inform your CLI which network to bootstrap to. We store and update these connection details on S3 for you to easily point your configuration to.
If you have followed the instruction above to remove your .safe folder before installing everything again, you should have no existing network configurations - you can confirm this with the $ safe networks command.
You can add a profile for the latest testnet which points to our S3 location, you can add using $ safe networks add:
$ safe networks add fleming-testnet https://sn-node.s3.eu-west-2.amazonaws.com/config/node_connection_info.config
Network 'fleming-testnet' was added to the list. Connection information is located at 'https://sn-node.s3.eu-west-2.amazonaws.com/config/node_connection_info.config'
Now you need to ensure you are set to use this fleming-testnet configuration that we have added, we can use $ safe networks switch fleming-testnet for this:
$ safe networks switch fleming-testnet
Switching to 'fleming-testnet' network...
Fetching 'fleming-testnet' network connection information from 'https://sn-node.s3.eu-west-2.amazonaws.com/config/node_connection_info.config' ...
Successfully switched to 'fleming-testnet' network in your system!
If you need write access to the 'fleming-testnet' network, you'll need to restart authd (safe auth restart), unlock a Safe and re-authorise the CLI again
We now have our CLI, Authenticator daemon and Node components up-to-date, and we have the latest hardcoded contact details to bootstrap to the public testnet, so everything is in place and we’re ready to launch our node and add it to the Network. This is achieved using $ safe node join as follows:
$ safe node join
Creating '/Users/maidsafe/.safe/node/local-node' folder
Storing nodes' generated data at /Users/maidsafe/.safe/node/local-node
Starting a node to join a Safe network...
Launching with node executable from: /Users/maidsafe/.safe/node/sn_node
Node started with hardcoded contact: 161.35.36.185:12000
Launching node...
Node logs are being stored at: /Users/maidsafe/.safe/node/local-node/sn_node.log
Your node will now launch and attempt to connect to the public network. Keep in mind that it can only join the testnet if the Network is accepting new nodes at that time (see NOTE below). You can keep an eye on its progress via its logs, which can be found at $HOME/.safe/node/local-node/sn_node.log.
If there is no space on the testnet for your node to join, as of Fleming testnet v2, your node will automatically try to rejoin until it is successful, or until you kill the sn_node process.
NOTE - please keep in mind that we have an anti Sybil attack feature in place now to only accept new nodes onto the testnet when resources are required, therefore you may be attempting to join the Network but your logs tell you that the Network is not accepting new nodes at this time:
Cannot start node due to error: Routing(TryJoinLater)
This is expected behaviour which will happen each time you try to connect until the testnet detects that resources are running low. As of Fleming testnet v2 your node will automatically try to rejoin in a loop, you do not need to attempt to join again manually, or via a script.
You can help to speed up the process by adding some data yourself - you don’t need to run a node to upload data! See the Do I need to run a node to participate? section below.
Before working your way through the CLI commands to perform various actions on the network, remember to authenticate, create and unlock your Safe. Following the steps in these links gives your Safe some test Safe Network Tokens to use. This means that there is no need to farm first to earn rewards before being able to try operations such as uploading to the testnet.
You can even connect to the testnet in read only mode using CLI, i.e. once you’ve installed CLI you can try fetching content uploaded by other users. For example, you can download the following image and open it locally afterwards:
$ safe cat safe://hygoyeye9mq5jmipo3we79wohue8hjyw55e6f8xyk1hquwy9hsxnk9tdsme > ~/safe-the-planet.png
Do I need to run a node to participate?
No, you can join with just the CLI and authd to experiment with data, tokens, etc. You can follow the instructions in the Joining the Testnet section above, but you don’t need to run $ safe node join - at this point you can just go straight to using the CLI as per the User Guide.
Further Information
Where are my node logs?
When you launch your node you should see the location of your log file printed on screen - this will be $HOME/.safe/node/local-node/sn_node.log. You can tail your logs with a command such as $ tail -f $HOME/.safe/node/local-node/sn_node.log
Where are my rewards?
If you’ve started a node you will have keys generated for you, stored in two files next to the logs of your node (see above). In case you want to access any rewards earned, it’s important that you copy these files to another location before starting a new node (otherwise they are overwritten).
The keys can then be used as per the instructions of the CLI manual, for transfers and payments. We will provide a UX and app for easier use of these soon.
I didn’t want to veg out on the sofa in front of a movie tonight anyway. haha.
Not sure I’m doing something wrong though.