I think the idea is that the size needs to be consistent across nodes using a distributed data model - that way the adults can know how full tings are just by looking at it’s own storage.
The small node size means less work for each node. So Elders only need to run one storage node. Hence no massive increase in work there and in fact less work overall maybe as don’t have so much communication to parse.
Churn is also possibly less of a problem with small node size.
The ideal size maybe a magic number of sorts, but can be approximated after a few test nets trying different sizes.
I wonder about deduplication though - with smaller nodes do the odds increase that a single machine running say 100 nodes may contain many if not all copies of a particular bit of data? Is this factored into smaller nodes?
A good point. It is possible but as the network grows more improbable due to distribution of data and node age forcing new addresses (so not able to sybil an address).
It’s like now with a small network and in our testnets somebody throws up 200 nodes, let’s them fill and switches them off. There is a chance, but as the network grows it becomes insignificant. It’s also a reason we don’t want mass node joins too mind you
Maybe post-beta, but maybe specialty nodes can come into play. So someone with a lot of bandwidth could offer to run a relay node. Having node specialties can allow for people to take best advantage of their hardware and connection. If we get Safe-processing down the line that could fit in there too.
maybe more copies at early stage or perhaps elders keep track of nodes run at a certain IP to prevent duplicate stores on one machine. Maybe another layer is added and each physical server gets a meta-node-id and runs as many sub-storage-nodes as it likes. Then data can simply not be duplicated on the same meta-node.
Striking the right balance for adult node size is a tricky one. Too large relative to bandwidth makes churns hurt. Too small relative to available cpu cores or ram in the host is asking for trouble too. I do think that uniform node sizes (50GB, 500GB etc.) whose requirement grows slowly over time is a win for stability though… Perhaps this could be built into the section split process. The section could either decide to split or double the per node storage requirement, etc.
You can have many nodes on one PC each with different IP, also you can have many different PCs behind NAT and only one IP. Then you have servers with virtualization and shared remote storage and so on. I don’t think it is worth (or even possible) to detect the “real” topology.
This is a whole network problem.
This is only problem for some nodes, not so critical.
In regards to reducing the chances of the network losing data, I don’t think these are significant concerns - few would be so configured in the first instance and the second instance wouldn’t matter.
I do prefer the idea of having a meta-node-id versus an ip-linked set of nodes though.
Probably my suggestions on this point aren’t really needed at all if David is right and such data loss would be insignificant.
I understand. And the “small” node idea isn’t bad. In fact, thinking further on it, I very much favor it for the short term because it’ll be faster to get the network up and further improvements can be made overtime from there if needed.
With that out of the way, the idea for the network to be agnostic of node size is to achieve the same end goal. But I left some things unstated. For instance, replication count with the large/size-unknown nodes can be much, much greater than 4 which would make churn from a data perspective less of an issue while maintaining its properties for sybil defense.
But the key future problem I’m anticipating for suggesting the idea is actually that bandwidth will become a limitation if the network is successful and data sizes become massive. Entities with large datasets on AWS understand this all too well, which snowball tries to solve. In a “small” nodes network, massive amounts of data would have to always be on the move, which could be wasteful and slow (has a hard limit at a point). Meanwhile, with “large” nodes with a lot more replicas, data will be longer lived in a section and bandwidth limited backup will happen in the background without being noticed much.
Again, this isn’t an issue right now and will only become of concern if the network is having to store massive amounts of data. So, a problem for the future.
This is true, all of it. I agree it’s all future. I wonder though will bandwidth always be more a limit than disk space. Or more precisely will we have many large non-churn nodes (i.e. nodes of age 100 will likely churn in a timespan measured in years). We may even end up with a 2 tier long storage layer with huge drive and short term layer with small drives for fast moving data and mobile phones or similar.
Or maybe a layer for small drives and small computers/phones that is paid to keep the bandwidth burden off larger and older nodes.
By the way, it seems this testnet was expected to meet an even higher bar from expected number of elder responses perspective. 7 out of 7 elder responses are expected, leaving no room for error. But normally, it’d be 5 out of 7 (so having two non-responsive elders wouldn’t have been an issue) or even lower for happy path.
So all things considered, a successful testnet I’d say.
I like this idea in principle, but given the high storage variance at present the estimate will be very loose indeed, eg 12G - 18G. otoh, if we could have a nice uniform distribution of chunks (as discussed above), then the estimate might be accurate to within tens of Mb.
Can you elaborate on how an attack would work if they did do that ?
How many “carefully-crafted” chunks would they need to assemble and then send?
I’d imagine it would be very CPU-intensive and time consuming?
Not beyond the capabilities of a nation-state adversary, of course or do you feel there is a real lower-level threat here?
@Southside I just see dependency on behaviour of official client here.
If developer want uniform distribution, it make sense for attacker to try making it non-uniform.
How many such chunks are needed depends on how sensitive to uniformity network operation will be. If it will still work even in completely non-uniform case, then attack makes not much sense at all.
If chunks hashes are “random” and there are N possible locations to store them, then you will need approximately N times more CPU power than when uploading them to random location.
not so byzantine given that all nodes are hosted by single trusted entity in a single data center. And also testing participants (clients) are probably all acting in good faith… just uploading and downloading data as requested per test params, rather than eg probing for vulnerabilities, ddosing, and more generally “attacking”.
I did try while the network was up to repeatedly check for open ports against one of the elder’s 12000 port with age 255, as well as send quite a bit of random data at that port (simple what if scenarios).
FWIW, it didn’t seem to have make any difference as the network continued to respond favorably to client requests from my end. Then again, this wasn’t the main focus of this specific testnet and these sort of tests weren’t conducted in a systematic and joint effort in attempt to bring down the network from multiple external hosts.
This one we don’t really know, which is what I’m speaking to. There are even good faith participants who are going beyond what they would likely be able to do if they had to pay. Then there’s the unknown of the silent participants haha
I generally agree with your point re:rigor though. But more protection might have to be in place to have better visibility of the experiment’s variables/assumptions.
What do you think of simply coupling a node’s storage requirement to a function of its age? This would cause the max capacity within a single section to grow in a natural way. It would allow archive nodes to naturally emerge, while also making it easy to determine storage capacity of a section by tallying node ages.