A bit of history
A lot of people probably still remember the change from persistent to non-persistent Vaults. It actually means the following:
Persistent Vault: Stores chunks on your computer even when you disconnect from SAFE. You can rejoin the network, proof you have a certain amount of chunks and rejoin the network in a certain group.
Non-persistent Vault: Stores chunks on your computer but everything is meaningless when you re-connect to the network. You’ll start as a new node again.
We’ve seen a major refactor in Routing called: Disjoint Groups. And the idea is that a group of nodes all share the same routing table as they are responsible for the same address-space.
Data chains and archive Nodes
What’s the purpose of upcoming releases before we can move to BETA? It’s a feature called “Data Republish”. Imagine a TEST net where we all upload our websites and some data, collectively disconnect our Vaults after 24H and start the network from scratch (with the same network-name) with all data 100% republished as it was when the network got down.
So the question is, how?? The devs are thinking about this for quite some time and 1 idea is the usage of “Archive Nodes” in combination with “Data Chains”. The idea is to link all the hashes of all the files together. Each group of Vaults knows exactly which chunks they need to preserve and they might even store parts of the Datachain outside of their group to get an image of the network in their neighborhood. Several archive nodes are allowed per group and they should store all the chunks the group is responsible for, being able to republish them when the network goes down and gets a re-boot. But is this the smartest thing to do in the short term?
The near future
I think we should go for the most simple solution for the data republishing problem. Farming isn’t active yet, there’s no incentive for any Archive Node to come back online for that reason. And why should we account on 6 Archive Nodes in a group of 24 nodes for example? It doesn’t add to security.
What I propose here is to start the network with persistent nodes only. At least until we’ve reached BETA. How could this be done?
Implement Node Ageing as a start. This will make sure the network is secured against a group-attack involving thousands of nodes. New Vaults will hop from group to group and all gathered chunks of data should be persistently stored on the harddrive of the user.
Implement Data Chains. This as a way to keep track of all the data chunks in the network as proposed.
When trouble occurs on the network (like a big outage) each Vault restarts and tries to join the latest group it was part of. If that fails it tries to go back to the group it belonged to before it was relocated. We could re-boot the network several times over to see if everything goes well.
Pros:
I guess persistent Vaults are way more easy to implement than Archive Nodes with all their roles.
All Vaults storing chunks makes data way more redundant. It gives us a bigger chance to have a successful reboot compared to only several “trusted” Archive Nodes. Why would we want to have this form of “centralization” in the first era of SAFE?
There’s always the option to implement Archive Nodes when the network is bigger and more stable.
It will speed up development of several App-projects (including Safecoin!) as the chances are way bigger that all data in the network is preserved.
Cons:
Having chunks on harddrive isn’t as secure as non-persistent Vaults.
Vaults coming back online need to re-sync with the network.
So here it is. Hopefully a discussion about persistent and non-persistent Vaults again
I wonder does that suggest bringing back older nodes first would see a better result… perhaps if old nodes knew to link to one another, they could coordinate accepting newer nodes incrementally until all are resurrected.
Reminds me of @bluebird and how he always worked so hard to get a persistent, community-run (decentralised) SAFE Network running at all times.
I always supported his work and now I will totally support this because a truly decentralised, always-on, community-run SAFE Network will help this project in sooo many ways, the least of which is that all people watching SAFE will feel so much more connected and involved (& farming practice)!
Would be so much better and engaging than a short, few-hour test here and there every few months. Just look at the gr8 press we’ve had with even that tiny short TEST 12!
I don’t know what you mean by older nodes? Let’s say you ran a Vault for 2 days. “Node Ageing” made you hop over 4 different groups. It is part of NA that you can always go back to the latest group, although it might cost you some ranking. This idea isn’t very different from that either. You try to connect to your last-seen group before the network broke. If that doesn’t work out (maybe that group re-merged with another group several seconds before the network crashed) you try the group you were part of before. You still have all the chunks so why not?
Interesting idea but I see archive nodes getting over shadowed long term as far as community testing because once persistent data or data republish is established, I imagine there would be less interest in small, short term, community testnets. Most will concentrate their efforts on the long living network and ignore anything that is a further step back from Safe coin implementation. Just playing devils advocate here! I think it’s a very cool idea otherwise
The idea is not only to have this implemented for community nets, but that it’s implemented until we’re in BETA. I just read it back and it’s not really clear from my post I see. But here’s the idea:
Implement “Node Ageing” and “Data Chains” before we move into BETA (as planned)
But also move to persistent vaults until we’re in BETA.
Otherwise the devs have to come up with a whole blue-print for Archive Nodes and questions about how many of these nodes should be allowed to a group and from which point we consider them “trust-able nodes” etc. To me that sounds like way more work than to move to persistent Vaults which store all the chunks and are able te rejoin the network again with all chunks included. And when we’re in BETA the Archive Nodes could be implemented if after all. But at that time we could have 2 millions users running BETA with persistent Vaults . We could even have Safecoin and Farming implemented before Archive Nodes are implemented. That’s my whole point. Why not let all Vaults be “Archive Nodes” to begin with?
I expect I don’t understand node ageing… if that is just pushing vaults into new groups, then that’s different.
I was thinking that if you have datachains, you have a time…however long that piece of string is… and then a sense of which vaults are backbone, which perhaps lends itself to stable recovery.
I think I don’t understand the option for a node that cannot find it’s previous group… and the risk associated with what data it held. So, at first pass looking for how at least one instance of data can be confidently retrieved.
A Vault joint the network (a group) but it doesn’t have anything to say as it isn’t really trusted yet. When 1 other Vault leaves this group, this brand new Vault is moved to another group. So it went from age 1 to age 2. Now it’s trusted just a little bit, it helps out a bit, stores some chunks. Now when 2 other Vaults leave, this brand new node is relocated again and gets to age 2. That’s the idea of ageing. The longer in the network, the more groups it will see, and the more it can farm.
Devs, Please keep safenetwork truly decentralized, We all hate and don’t need or want to trust any node.
If the network will not be 100% truly decentralized, Then the entire project will fail.
. We could even have Safecoin and Farming implemented before Archive Nodes are implemented. That’s my whole point. Why not let all Vaults be “Archive Nodes” to begin with?