Today, we started testing Test 19 internally. As mentioned last week, we are continuing to work through JIRA tasks and fixing issues that need to be completed for Alpha 2. The network itself is looking good in internal tests and we don’t expect to make major changes to Routing & Vault before Alpha 2. However, there are still a few front-end issues we need to fix before releasing Test 19. Here are some of the issues we intend to address in the next few days:
- MAID-2288: Uploading a directory with many files fails in Web Hosting Manager
- MAID-2289: Permissions list in Authenticator always shows a full list of permissions
- MAID-2270: Update Web API playground tool with changes in containers permissions API
- MAID-2290: Show empty log page / log size
- As well as other minor issues and UI changes
We initially hoped to get this testnet out tonight, but we feel these pending issues might affect general UX quite a bit. We’re hoping to have these resolved quite soon and get the binaries out to you to test these new features. It is fantastic mind you, to not be waiting on backend, vaults and routing etc. as these bugs are annoying but not keeping us awake at night. We do feel the pain of delays, probably more than most and we are desperate for the energy test network releases give us all. While backend libs are stable and Alpha 3 work continuing from the guys there, Alpha 2 and these pending front-end tasks remain our highest priority to complete as soon as possible to get the new test network going to keep progressing with the roadmap.
Recruitment Continues
As discussed in previous updates we are growing the (local) team across areas including engineering and marketing. We are also now looking to recruit a Help Desk Manager and a Release Cycle Manager to aid with the ongoing rollout process.
The Help Desk Manager will plan, set up and run a help desk (using JIRA) as the network advances through the up and coming test iterations toward beta and beyond. They will also record relevant information from test networks and compile update reports, as well as identify trends in support requests and perform root cause analysis.
The Release Cycle Manager will be multifaceted and cover quite a lot of areas including a combination of: thoroughly testing all new releases, ensuring that all risks, impacts and dependencies are all clearly understood and communicated. They will also write, maintain and manage all the technical documentation, something that we all appreciate needs worked on. We also envision that the successful candidate would get involved in helping to showcase and demonstrate new feature releases to all stakeholders.
What we are looking for with both roles is a very proactive approach to helping all users of SAFE software and this is another important step in positioning the company to be more outward facing and commercially minded, and to garner ever more feedback from users with a view to improving our future releases.
SAFE Authenticator & API
The main focus and effort during the last week on the front-end side was finalising the integration with some changes required in the safe_client_libs and safe_app_nodejs APIs to enable the sharing of MutableData which impacted several artifacts including the UIs and UX of the applications.
As explained in the previous dev update, both the Web Hosting Manager and email app now support sharing the same public ID for creating/adding email and web services, regardless of which of them created the public ID. This implies an authorisation request sent for the user to allow/deny sharing the MutableData behind the services’ container that is associated with that public ID. Internal integration tests of this new flow in Test 19 are going as expected and no issues have been discovered so far. The authorisation pop-up in this scenario displays the MutableData’s metadata to the user, but since this information is not enforced to be set at creation time, it can be empty. Therefore, in order to help the user make a more conscious decision at the moment of authorising a MutableData to be shared, the Authenticator has been enhanced to not only display the MutableData’s metadata information, but also to show the list of applications that are already authorised to access it, which includes the application which originally created it. The tests so far are showing that this functionality is also working as expected.
In parallel to the above, safe_app_nodejs and authenticator were adapted to accommodate to enhancements made in the way that the containers and MutableData permissions are retrieved through the APIs, making it more homogenous. One of this enhancements is to be able to retrieve not only the list of containers that the application has access to, but also to get the set of permissions the application was granted for each of the containers. This forced us to rename one of the APIs exposed by safe_app_nodejs and the DOM API, from getContainersNames to getContainersPermissions, to make it coherent with its semantic.
One of the other goals for the last few days was to try to work on some minor issues reported by the community either on GitHub or on the forums, and several of them were fixed or verified to be already solved, while others are being actively reviewed at the moment, while we also try to provide as much of a help as possible to the developers asking questions.
As mentioned in the introduction above, the main focus for the next few days is to solve a few issues that we prefer to close before sharing the new binaries with the community, as well as perform more tests to make sure that all the pieces are working fine together; in the front-end there are several integration points and covering all flows is not a trivial task.
SAFE Client Libs
This week, we’ve been focused on fixing known bugs and reviewing existing functionality in the SAFE Client Libs. We started with issues that were mentioned in some of the previous updates: we suspected that requesting app revocation from several Authenticators simultaneously (e.g. from several machines at once) wouldn’t work, and this was a known limitation. Testing this feature is hard, as we need to simulate this behaviour from several processes or several threads, which are not deterministic - that is, we need to run the tests multiple times to verify that they’re working as intended and there’s no undefined behaviour. @ustulation outlined the test scenarios in the JIRA task MAID-2235, and it’s been implemented by @adam in this pull request. The test results have shown that parallel revocation was not working due to some minor errors in the flow, and these bugs are fixed now by this PR (which is currently being reviewed and will be merged soon).
We also discovered another obscure bug that could result in the mock-vault data corruption (it doesn’t pertain to the real network). In some cases, after a user deleted an entry from a Mutable Data object on the network, authorised apps could not load anything from the mock-vault, so it was deemed corrupted, as described in this bug report. But as it turned out, the case was a bit more involved: we didn’t truncate the file contents before writing to it, so there could be a case when some junk data remained at the end of the mock-vault file. This is now fixed by this pull request.
In the mean time, @marcin worked on adding more tests to our suite, verifying that existing functionality works as designed. For instance, we have confirmed that the complete removal of revoked apps is working fine and that sharing access to arbitrary Mutable Data objects on the network cater for edge cases. Apart from that, @marcin has implemented a new API function to fetch a list of apps that have access to a certain Mutable Data object on the network. Other minor improvements include bug fixing in the SAFE App library (the file_close function didn’t return an expected value) and refactoring of access container functions (which are now made more flexible and generic) - both changes are landed in a single pull request.
To conclude: we published a new version of the system_uri library (the changes optimise apps IPC communication on Windows and Linux) and released our core libraries, safe_app and safe_authenticator, as version 0.3.0, and safe_core as version 0.26.0.
Routing, Vault & Crust
Routing had version 0.33.0 published, which contains the major features of improved rate_limiter and couple of bug fixes, as described in the previous dev updates. We are also investigating a new swarming mechanism, and aiming to make it not only secure but also avoiding the complexity of O(N^2). As an additional benefit, it is also being evaluated whether it is possible to reduce the number of connections a node has to maintain. This would allow larger sections, which improves security.
The Vault PR to integrate with the latest Routing is finally merged. This utilises the new Routing rate_limiter features: soft capacity and resending parts. The previous re-sends on exceeding limit carried out by the test_client is no longer required. And the new version of Vault, 0.17.0, also got published.
Crust has been getting some love again this week. One noticeable improvement is that it now uses file-system notifications to update its configuration whenever its config file is edited. An effect of this is that if a node has a configured whitelist of peers, changes to that list will take effect immediately, dropping connections to de-listed peers and allowing connections from newly-whitelisted peers. There has also been a small amount of general refactoring in Crust, which has fixed a race condition which would occasionally cause our continuous integration tests to fail. Additionally, there’s been some small bug-fixes to Crust’s test suite.
's / love 
