So I was thinking about network design and how GETS are free on data. Please tell me how we programmatically combat this:
I am Farmer A, I have been doing SAFE for awhile and hosting a hella lot of data because the network is so popular now! All my storage devices are now full and I want to continue making safe coin by storing new files. Therefore my next action would be to wipe my disks clean and reinstall the software and free up all that space for new files.
This is concerning because it would mean older data gets limited to a few huge older vault nodes that would be willing to host vast amounts of old āunprofitableā data most likely.
Am I missing something here? How does the network handle this situation without incentivising the GET of older data?
Your rank will go back to zero and you will have to re-earn trust before you receive lots of data to farm.
If you are genuinely maxed out, I suppose this may still benefit you. It will also have the positive affect of dispersing the data to other nodes which have available storage. However, if you still have lots of space, losing rank will probably cost you more than you would stand to gain.
I disagree that it would be a positive affect but really hope to be proven wrong , I believe it will cause centralization to vaults willing to host lots of old data that have the capacity. Will be interesting to see in practice but I believe this will become a potential issue with this networks design.
With so many competing actions, we will only know when the network goes live. However, I suspect rank will be weighted sufficiently to discourage excessive data abandonment in favour of adding more storage.
Right, if its a harsh network penalty to ādump diskā like that because node age has gone down then that may suffice. Another idea I had would be how possible could it be that the network recognizes with fair confidence the āfresh vaultā is back up and proceeds to store most of their old data right back into the node with how the network heals itself and replicates data that has fallen off certain nodes. Then who cares about penalties, even a slight one would not be worth it if they knew the encrypted data would mostly be flowing right back to them. Something to ponder at least, I try to poke holes in the high level concepts daily to keep myself honest and not turn into a fanboy . Thanks for sharing some insight though, great talk!
The likelihood of that is very small, a node will randomly be put in a section on the network after its age is halved for failing to act properly. It cannot decide on the section, but if it tries to just reboot and join again, its lost another 50% of rank and on that process goes. When it has a lower rank it only stays so long in a section (proportionate to its rank). If it loses enough rank it wonāt even be an elder and will get paid much less safecoin.
Seems tough, but the network should aggressively deal with non-compliant vaults, the flip side is that new vaults will find it harder to initially earn safecoin though. Obviously when the network starts most nodes are very new and will earn quickly, as it grows then getting to become an elder in the section is harder which seems pretty natural. Established groups/herds etc. make it tougher to join, but offer a lot of safety when you do etc.
I think as we progress and storage keeps scaling at the rate it is, then the current solution does make the most sense because vaults could start holding petabytes of data no problem in a few years. The only thing that pops into my head currently with storage sizes today in mind is that if a vault is full and has no way to simply add capacity would turn to dumping because they canāt earn safecoin anyways so any penalty is better than not earning(if you think in terms of people, most will not do a service for free when they can get paid for the same service. Granted we have plenty of loyal fans here that would run vaults for free just to support the network, those will be few and far between when the network reaches global adoption though ) . But overall I think the approach taken now is a pretty good one. Thanks, definitely helps!
I donāt think the next action is rational. The rational action is to use the earnings from the existing vaults to commission more hardware to run more vaults and store more data. Throwing away vault rank / age is like throwing away āworkā. Accumulated work has value on this network.
An interesting thought experiment: what if instead of storage space itās bandwidth that the operator runs out of? How do they resolve that?!
I agree this is a concern. The idea that āsome data is more valuable than othersā is toxic to the idea of permanent storage.
There are quite a few ways to combat this, but itās a bit of a long discussion for another time.
Maybe itās ok to have a āvault class systemā where some vaults do the high-volume-short-lived āmeme trafficā and some vaults do the low-volume-long-lived ābackup trafficā. So long as the rewards match the goals it will be ok.
Or maybe vaults should be identical and all vaults do a bit of every type of work. Weāll see after alpha3 and the design work for incentives begins.
Itāll be interesting to see how ādump diskā or even āunpopular chunk abandonmentā can be detected by the network.
I think vault operators are incentivised to run many small vaults rather than a few giant vaults because when relocation happens they will have less data to churn.
But on the flip side large well established operators want to have big vaults because it excludes new competitors from joining. Itās an interesting dynamic that will need to be managed by the incentive structure of safecoin.
Actually one of the future enhancements is to have archive nodes which do exactly this, they collect (are given eventually) the data that is accessed the least. They earn because even the least accessed data will be accessed and the volume mean reasonable earnings.
The idea is that there will be 1000ās or 10000ās of these around the globe run by whomever wants to and help to ensure that the more used data is in other peopleās vaults. These archive nodes are not all that special really and more like what you suggest, they are extra large nodes that remain on virtually 24/7/365
This is basic and all the analyses must start from this fact.
1.-The amount of data that a Vault can store will always have a correlation with the rest of Vaults in its section. It is not probable that one vault can store much more data than other elders of their section.
Also, the amount of data of a section will tend to be similar to other sections of the XOR space so a vault bigger than the rest will be, possibly, wasted.
2.-The growth of the network will lead to the continuous split of new sections, so the amount of data that a Vault must store will not grow at the same rate as the network.
3.-If we consider that the network may be growing for several years, most of the data will be relatively recent (possibly less than a year). Even in a consolidated network, and considering the data growth of the last decades, always new data will be most of the total.
But Iād add that it is probable that some people will try to manipulate the average amount of data stored per vault.
Some people will probably try to reduce the average storage per vault (eg by starting many vaults) so thereās less data to transfer when their vaults experience churn (among other reasons).
Others will probably try to increase average storage per vault by whatever means they can to squeeze out the little guys.
This āfightā for average storage size needs to be mediated by the safecoin incentive structure.
I for one would prefer to run many smaller vaults since it gives me greater flexibility in managing my resources than a single large vault which is a binary on/off choice.
Yes, possibly. It depends on how the incentives are established. Currently safecoin is designed to encourage some spare capacity. However there is a balance point between vaults providing spare capacity vs using that spare capacity to run more vaults > store more chunks > serve more GETs > earn safecoin. But presumably there will be an optimum amount of spare space and all vaults would run at that, so your point about it being a waste to run vaults with too much extra space is still very relevant.
I agree with your points but Iām not sure whether it concludes vaults will tend to aim for small or large average storage per vault.
Safe is inspired by nature and therefore will follow the same laws and those that best adapt will survive.
As the greatest punishment occurs when a Vault dies, although its resurrection is allowed at the cost of losing rank, those Vaults that live longer will dominate. As those located in datacenters are more likely to survive everything indicates that large and professional Vaults will eventually prevail although, possibly, it will be a rather slow process.
However, I have great doubts. With the SAFE network we enter a totally unknown territory where new hardware, communications or developments can change the future completely.
Reiterating on your points (mostly just saying the same thing):
Yes, this should be it. A new node will get data from the section, in case there are any chunks there with lower replication than required - not otherwise, right? Distribution of incoming new data within a section should be equal over time. And distribution of new data over sections tends to be even too.
As new vaults join, sections split and likelihood for data stored to reach a specific section decrease, so previous vaultsā share of new data decrease.
If vaults count increase while data inflow does not, then rate of new data to a vault decrease.
Data stored in world increases all the time. Growth is exponential now. Until there is a downturn in human activity this will continue. There will of course come a downturn at some point. And at some point data storage costs might not reduce more (could come very large drops at certain threshold breaches before that though)
But up to any of those points, old data will always seem relatively small.
So itās interesting to see how this looks at individual vault level. @mav, how about dynamically increase vault space? If I manage to keep a machine running with power backup and no OS upgrades, and connection is super robust, and so on, I could maintain higher node age and just attach more disks to never be filled up. So it should be desirable to enable a vault to include that space without restart.
There will be a race to attach more space, because new data will be bigger than old, and the access attempts to old will rapidly decrease.
Keeping the old in there will not be a problem then it seems, as it will āalwaysā be relatively very small compared to the new data, that you are allotting new space for.
This is one factor, but there are others that push in the other direction so there will be a balance point, and also the network can be adjusted, tuned, to account for undesirable effects.
One factor is cost. Running a vault in a data centre has ongoing operating costs: renting the servers, paying overheads (if this is a business) and making sufficient profit to be more attractive than other uses of capital. And in the scenario described this has to remain so indefinitely.
Also, I believe that the advantage of age tails off, so being around for twelve months may not be that much better than six months, if you see what I mean.
Pushing in the other direction, individual vaults on legacy hardware have little to no discernable costs, and can join and stick around without concern for shareholders, sunk costs etc - joining and farming when rewards are high, sticking around or leaving when they are not.
So I see balancing forces, which is what we need to avoid extreme centralisation, and there will be the option to adjust these forces to achieve sufficient decentralisation.
, I believe it will cause centralization to vaults willing to host lots of old data that have the capacity. Will be interesting to see in practice but I believe this will become a potential issue with this networks design.
. Thanks for sharing some insight though, great talk!