Dev Update - Week Beginning 21st July 2014

Hi Everyone,

Its that time again, i know you all are desperate to see TestNet 2 up and running, however the best things come to those who wait, and believe me its well worth the wait. The good news is we have just started a recruitment drive which will entice the best coders around to join this game changing project which will hopefully speed things up a little.
Anyway lets start this week off from a QA’s perspective:


This week involved a lot of TestNet 1 testing, you may have noticed that the visualiser was not as stable this week as normal as we had to bring it down a few times to implement new code changes which is a good thing :slight_smile:
On the visualiser, you may also notice that there is a TestNet 0, this is vaults that are running locally and for local testing purposes, these vaults are running on odroids in the office and next week we will start the process of running a nightly local test network running various tests that will output results onto our dashboard:

TestNet 1 Store/Get data chunks | Network drive | (WIP)

Further improvements to self encryption library. Drive sanity checks going well (thanks Bjorn). Possible change in encrypt/drive API to reduce memory allocations and increase speed significantly. I expect drive to be fully capable on the network in the coming weeks.

TestNet 1 : Network drive (WIP)

Finally we agreed on the bootstrap cache file location and properties. This is implemented now and under test. This slowed down Example 2 and we felt Viv’s frustration. We also allowed for users to be able to pass IP:port combinations around (not implemented this week). This means that even if the backup bootstrap nodes were compromised or failed (or worse) then people could send each other bootstrap points over the phone or email etc. This would also work for extreme network segmentation (disaster zone etc.). This took too long, but typically it is these seemingly small issues that require careful thought, so bootstrap cache version about 1 million is now agreed.

TestNet 1 - Setup Seed network, join network

TestNet 1.V6 has run over last weekend and till this Wednesday. There were some nodes from all four geo sites had problem in joining the network at the beginning (35 out of 212). The debug work then carried out and some fix has been applied to code.

Later on Wednesday, TestNet 1.V7 was started up. The fixes proved to work as all nodes from low latency geo sites joined network properly. Nodes from long latency site (Singapore) having problem in join solely because of the time out.

Both TestNet 1.V6 and TestNet 1.V7 proved once vault joined network, it is very stable in keep connected to each other.

Below are some additional work that has been implemented this week.

  • Fixing routing infinite re-bootstrapping while routing table is empty.

  • Debugging Put / Get operations. In particular get_handler used to keep unnecessary entries for retrieved data. Both put and get operation used to expect wrong number of responses.

  • Debugging some issues in TESTnfs_func – some tests used to show success in case of both success and failure.

  • Removed NodeId::MaxId completely from routing library.

  • Debugging a routing segfault  raised by Brian, which ended up to opting out clients from doing unnecessary operations.

And now for a TesNet 2 taster!

TestNet 2 - Obfuscate chunk on disk (complete)

This was carried out by Brian several weeks back. Now on storing a chunk, it is again encrypted and the name obfuscated. Only on a request for data can a machine actually remove the outer layer encryption (so chunk still encrypted as per self encryption). The request for a chunk will contain a hash, this is hashed again and the chunk found, it is then decrypted using the original hash and returned as an immutable chunk as per normal. This disallows any attack where an adversary may store bad data directly. This attack is described in the Attack section of systemdocs.


Two days this week were spent with Niall and Bjorn in the office, so David, Fraser and Prakash were involved in these meetings. This means a slight reduction in output this week, but the meeting was extremely valuable and allowed the team to re-align around current tasks for launch.

Encrypt was hindered slightly as David lost 2 days work with a hard drive fault on his laptop. (knowing David I am sure he has already made up for it :smile:

Well that’s us for this week, i hope this tickles your taste-buds for another week until then…




Thanks for the update!

1 Like

I share the disk failure pain - the same thing happened to me last week.

Thanks for the update! Sounds like good progress is being made.

1 Like

We need the SAFE network so these hard drive faults don’t happen :wink:


Doesn’t look like Ross has updated the road map in 22 days…is there anyway we can make sure that the road map stays currently updated? I think this would be an important confidence-building practice.