Hi everyone,
A few hours ago testnet1 was started for the first time; watch it come to life here - it really is a thing of beauty. This network will be up and down an awful lot this week, as we push, prod and poke it, so please expect to see it in all sorts of states.
To paraphrase @dirvine :
A ton of very hard, dedicated work is happening just now in the office, this is the start and it is happening with still some vital work to do in rudp and a known bug in routing which we will fix this week, even so it works. We are absolutely flat out and pretty exhausted so forgive us for any delays in mails and responding to posts etc. This week will see the start of the snowball rushing down the mountain and gathering pace. Exciting times!!
Here is some of the detail of what the team have been working on…
RUDP improvement - Testnet1
This week finally saw the beginning of actual changes to RUDP that aren’t bug or race fixes. As had been noted in last week’s update, the addition of constant and bursty packet loss simulation to RUDP had shown that RUDP is unstable under packet loss due to the connection bootstrap code being packet loss intolerant (once a connection is established, packet loss is tolerated very well). Indeed, connection bootstrapping is so intolerant to packet loss that in a few places if a certain single packet doesn’t arrive intact and quickly, the connection is killed - not good for Wifi connections which typically see constant packet loss. The ability to retry packet sends and to tolerate duplicate packet receipts is being added to RUDP connection bootstrap, it is hoped this might produce a working RUDP sometime next week.
Mobile platform integration
MaidSafe is already daily continuous integration tested on oDroid ARM development boards, these run a quad core 1.7Ghz Exynos 4412 ARM Cortex-A9 CPU with 400Mhz RAM and are representative of a two generation old smartphone (e.g. Galaxy S III). Today we received a Tegra K1 development board ( Jetson TK1 - eLinux.org ) which is a latest generation quad core 2.3Ghz ARM Cortex-A15 CPU with 933Mhz RAM and which is a considerably more powerful test target capable of running an unrestricted set of unit tests. It will be integrated during the coming week into nightly unit soak testing with as many of the correctness and sanitiser tools as are supported under ARM Linux.
Sanitiser fixes - Testnet1We have been working on sanitiser fixes in Drive and Encrypt. Encrypt is nearly there, Common and Passport are now sanitiser clean and Encrypt is undergoing some work in exception safety. This affects Drive quite a bit. All of this is very nice to have and does not slow down launch, but we like very clean code.
Nfs - Testnet1
Very quickly we will move on to an Nfs FileSystem API that will allow reading writing raw information to a posix like interface. This will give incredible flexibility to app developers as they can then basically do anything they wish and treat the network like a large hard disk at a very granular level (like Read(char * data, int32_t length, int32_t position) etc.). This will also allow multiple language bindings to be created very quickly. This will happen during testnet1.
Meanwhile, NFS chunk level functional tests have been created / expanded to test against vaults network (a local network or eventually testnet1). Such test sets aim to simulate user behaviour from chunk level: storing / getting data, updating versions of SDV.
Routing - Testnet1
Routing got a major bit of work this week to remove a large container (group Matrix) as we managed to ensure greater accuracy and speed via extending the close node group to encompass any errors in further away nodes (essentially we take a group of 16 to be sure of 4 of them, tests show we can be sure of 5 actually). This delayed the testnet launch this week really, but is done now and is now on the third day of testing, with scaled network tests to validate these group matrix changes.
Bootstrap file handling - Testnet1
Bootstrap file handling is now implemented at routing level using sqlite3. To achieve this we have integrated sqlite3 into our build system and wrapped the required sqllite3 APIs behind exception safe c++ code. We further aim to get the sqlite3 test suite integrated with our Continuous Integration tests.
Bootstrap file handling changes will allow multiple processes to access the same bootstrap file concurrently and now we are working towards adapting upper layers NFS/Vault/Drive/API to this change. As this is an API change, we are working towards updating a lot of tests in all the affected libraries. This is the last missing link to auth example and hopefully after this we can have the auth example out.
Network drive - Testnet1
Network drive and its tests have now been updated in line with the latest changes and we are working towards fixing test failures on different platforms.
Testnet1 - approach / preparation
The automated mechanism to quickly (relatively) roll out the infrastructure and software required to build the testnet was completed and tested. Also the opposite process required to destroy the same network has been completed and tested. This will allow us great flexibility when testing various scenarios.