Testnet cognoscente will know that while some iterations represent a decisive step forward, not all deliver the goods as planned. Alas, the latter was the case with the last beta iteration. It was not all bad by any means - we saw a reduction in messaging bandwidth of 15%. But it sure wasn’t perfect.
Beta testers waiting patiently for nanos will have seen their earnings drop to a trickle. There are three reasons for that. First, we have more people pre-joining Wave 2 through the referral scheme - which is a positive. Less welcome, however, are some interconnected issues that are slowing data uploads and preventing some home nodes from connecting properly.
In part, this is because we have had two testnets running in parallel, and they have “crossed over” in unexpected ways. This is (at least in part) what’s behind the slow uploads, as they are being delayed by verification checks.
The other issue causing the delay is libp2p relay which allows home nodes (nodes behind routers) to join. In short, the relay nodes are very conservative in permitting relay connections. When many nodes try and connect to a home node via a relay, the relay runs out of available connections and the home node becomes unreachable. Our attempted solution is relaxing the limits that relay servers have configured. We have an internal testnet to check this out.
The mixing of testnets also caused some turmoil with auditing Beta earnings and updating the leaderboard, which is a semi-manual process (the Discord Bot is actually cyborg, with some human components).
Glass-half-full types can take comfort from the fact that the network proved resilient even when unintentionally running two versions of the protocol. However, even the rosiest of rose-tinted glasses can’t conceal the fact that this hasn’t been a great week for brand Autonomi and our noble band of testers.
We can only apologise for the minimalistic comms. It was a perfect storm type situation where an unexpected issue with significant knock-on effects arose at a time when we were understaffed.
So, what can we do to make amends? Well, we’ll be taking the older testnet down shortly and have put some fixes in the audit DAG. We’re also investigating why the crossover happened in the first place. Most importantly @rusty.spork (yes that is his real name, although occasionally he goes by Aaron) is now on hand to provide prompt and transparent answers to your questions on Discord.
And, with the proviso of the latest tests passing, we’re also about to launch Wave 2 for real, meaning that everyone’s nano-earnings (remember, these decide your leaderboard rank and payout when the network launches) will be boosted.
That’s all for now folks - thanks for everything. 
General progress
@rusty.spork has been collating a list of common issues from the community, including the Windows vcruntime140.dll error (which requires the installation of Microsoft Visual C++ Redistributable), executables being blocked by Windows Defender or other antivirus software, and leaderboard delays. He is also looking at the stats for nano payments, finding that although the rate is increasing, only a small proportion of nodes are being paid due to the issues mentioned in the OP.
Our other newcomer @mick.vandijke liaised with @happybeing to help fix a bug whereby uploading a directory fails due to excessive payments being deducted which repeatedly drain the wallet. Obviously worried that he’s not busy enough, @happybeing just found another bug with folder uploads. Gee, thanks Mark! Mick also worked on the graceful shutdown of CLI actions, and on an issue with the wallet balance in the CLI (where it was occasionally reporting balance as the sum of all stashed wallets). Plus he’s created an option to save a password-encrypted wallet private key inside the wallet directory. 
@anselme continued his work redesigning spends to make them lighter, more secure and more efficient.
And @qi_ma is looking into how the two testnet versions overlapped. He worked on a PR which should at least mitigate the effect of testnet mixing in the future #1968 and did some fixes on the auditor to improve handling of this type of incident.
Meanwhile, @bzee is looking at our client API design, rationalising error messages and ensuring it is up-to-date, and removing legacy functions and arguments. He’s also been working on the relay issue, which is constraining home nodes (see the OP), and he updated the Discord Bot to get some of the commands working.
@roland is also looking deeper into the libp2p relay function. It’s not easy to tell which connections are relay connections and which are for other purposes. This is what Roland is trying to ascertain.
@mazzi has been working in conjunction with @bzee and @ansleme on priorities for the network API, so we can get a bare minimum of functionality ready for launch. He also made changes to the DAG to fix faulty or failed auditor services.
While bandwidth has improved, the network is still seeing excessive messaging and CPU usage. @joshuef put in some fixes to change when chunk verification is triggered, and to reduce a node’s compute requirements on receiving replication messages. Josh is also working with @dirvine on Sybil prevention.
@shu continues to refine monitoring so we can compare testnets.
Fresh back from holiday, @jimcollinson organised a Discord Stage with @bux to run through the issues summarised here. The two of them are also working on launch materials, marketing, and conducting talks with partners and like-minded organisations.
