I can’t see a perfect solution but I kinda feel that’s just kicking the problem down the road.
We envisioned a network that anyone could participate in, making sure it would run on old hardware.
I dont see not allowing nodes to join as they have slower internet than other competing nodes to be helpful to allowing anyone to participate.
2 Likes
Are there other areas where we face this? I’m sure there must be, so maybe the issue is how to deal with those rather than the question rolling this particular approach out.
2 Likes
Yes, I think so.
1 Like
I’m only brainstorming. The main idea is to use write failure mode directly as the provisioning driver, and concurrent duplication to give users great performance despite occasional internal failure, with the write still succeeding.
1 Like
With the randomness element. The fastest farmer(s) will always be the ones distributing the fetch but they will not reap all the rewards. A random element (lottery?) will send some of the rewards elsewhere. The network needs a way of estimating the amount of “uber” setups making up the network at all times. As the measure of uberness goes up the randomness awards go up. When the amount of uberness goes down, the randomness goes down.
1 Like
My instinct - and I know and have thought too little to be specific
would be for
- elders check how many nodes are full (80%)
- if it’s more than a certain threshold (50%) more nodes join the section
but also that other factors might want to be considered… disk space is one issue of performance but what of nodes that are slower to respond [sum of latency of bandwidth or the hardware] - if there is a test in place for responsiveness to perhaps the simplest kind of responses that it naturally gives frequently, then that could be used for adding in
- elders check how many nodes are full (80%)
- elders check how many nodes are slow (80% slower than the %20 fastest instances - or similar flexible responsive measure of the spread)
- if it’s more than a certain threshold (50%) more nodes join the section
That would give option to dump nodes which are becoming a problem for being too slow?.. or reallocate and share those to other sections with faster nodes; so, the network is balances on the capability of the nodes it sees in each section. Perhaps the overhead is not worth the benefits for some of this??
1 Like
The randomness helps ( I think ), only after the node is allowed to join, so no help to a node trying to join that is always beat by a faster competitor.
Or are nodes attempting to join also in the reward pool?
1 Like
Yes, a real lottery. This would go a long way to encourage wide adoption by all devices. The trick, as always, will be to get the random algorithm correct. Giving away too much of the reward in the lottery will discourage fast farmers from participating thereby slowing down the network, giving away too little will not give sufficient incentive to low-resource participants. A workable formula can be arrived at, though, I firmly believe.
3 Likes
This is why I brought up the idea of natural selection and introducing a RND function to the parameters. If there are ways to measure things from within the network and there is a means to curtail rewards and shrink sections that aren’t as successful, then the network finds equilibrium on it’s own.
But I guess that for now at least that’s all too difficult to logic out.
2 Likes
Why limit network growth?
Seeing more nodes, surely is better for stability?.. the OP seems to be relative to concern about farming pools and then some sense of adopting nodes where those are needed.
I guess perhaps there is a problem of attack from the flux of new nodes turning on and off… but if earning starts only after a certain age, that could counter this?
The risk of too many nodes joining perhaps is the reward for everyone drops but does that matter if the offset is that everyone wanting to participate is happy - and encouraged. The network then has stability and evidence of plenty of capability to adopt data, that would be an attractor for users to have confidence too? I can’t see if there is a inverted attack from farming spawning many small nodes but if that was possible equally it wouldn’t bring reward… if size was a factor in whatever method the network uses to encourage the kind of nodes it wants to see.
How does the network encourage the kind of nodes it wants to see?..
2 Likes
I think what you are looking for is a weighted roulette wheel approach here. Some AI algorithms use it, … in fact here is a link Fitness proportionate selection - Wikipedia
2 Likes
Perhaps read/write faults could increment some value (u8) and when it reaches MAX the vault ‘dies’ and is put back into the provisioning queue?
Combined with Proposal for network growth based on node size rather than node fullness - #24 by anon57419684 you get:
- Write faults drive provisioning of new vaults.
- When vaults accumulate a certain amount of non-performance introduced to the network, they’re killed off.
Continuing, I think latent state is a potential problem for a long-running decentralized network and having a mechanism for regular cycling of vaults through life and death could be good.
3 Likes
Basically attack vectors, esp in early days. The attack is mass join a ton of nodes, get them promoted up the chain and take over. If we accept any node they can then choose their section and target it. Limiting means we spread nodes in space and time, this makes attackers lives miserable and we want that.
14 Likes
Yes, that looks good. Actually, I think almost any way to arrive at a random outcome, or rather random outcomes, would work. Whatever would be the most expeditious route.
1 Like
Would a solution be both options?.. that only the larger and more powerful can become promoted… not necessarily the very largest and very most powerful as that would prefer those who can afford extremes but some cut that does away with volume of small nodes.
1 Like
I think we should always keep in mind how we plan to be attractive to individuals in countries where owning a cell phone is about the only way they are going to be able to have internet access, and how we can provide them with the possibility of rewards, even with their modest device/connection.
5 Likes
As long as best performance always wins, so improving local infrastructure is incentivized.
2 Likes
No, the steam data is just as an example. I’d like to see mobiles as nodes. If battery life is improved enough to sustain the workload it makes perfect sense to include them because phones are almost always turned on and connected 24/7.
Faster nodes don’t get priority. All nodes are still allowed to join. But the average amount of data stored by each node is determined by the average replacement speed.
I see where you’re coming from though. It’s tricky to balance performance with broad participation, especially in an organic way that doesn’t require predefined thresholds (as dirvine says ‘picked by who?’). I feel like node replacement speed is a better variable for measuring risk than node fullness. But I can see this introduces a parameter (replacement time) where fullness does not require one.
Maybe we could calculate it / engineer it, based on the risk. On one hand we want replacement time to be fast so chunks spend as much time as possible fully duplicated. On the other hand we also want replacement time to be slow so we don’t require elite hardware to participate.
The further I try to justify node replacement speed to control growth the less strongly I feel the benefits. But I do still feel the risks of using node fullness for growth are quite substantial. So I’m kinda on the fence with it, I’m less certain now than 35 posts prior. The conversation has been really great so far, much appreciated.
I wonder about whether we can set a ‘live discovery’ a bit like Tyler suggests with evolution.
Monero has a great concept where the parameter (in their case blocksize) can be changed by the miner sacrificing some of their reward. So miners have the opportunity to get some long term benefit for some short term cost.
I was hoping node replacement time is agnostic to technological development. Replacing a node within five minutes now is equally important as replacing a node within five minutes on a network at any point in the future.
Searching the codebase for const shows a lot of stuff… the main one that comes to my mind is 1 MB max chunk size, it seems quite arbitrary to me but being fixed and predictable also has benefits.
Just thinking out loud, I wonder how we can determine ‘too slow’? If it’s the slowest x% then that leads (via compound interest type erosion) to extreme exclusion. I feel like ‘too slow’ is pretty natural to express within the node-replacement-time concept. Not sure though, just thinking out loud.
I’m about 95% feeling we should try not to limit network growth, but 5% still sits at the back of my mind for security / durability / governance reasons. I feel radical inclusion is necessary at all levels - storing data, reading data, running a node, developers, payments, on-ramps, content moderation, content creation, the whole thing.
Any thoughts on how regular this should be? I agree with you and I slightly prefer features that promote node neutrality rather than elders-being-super-important.
Another reason why small nodes are better. More potential participants to take action if trouble is noticed. Would be a shame to see twitter lighting up with ‘Safe Network is under attack’ and the audience unable to do anything about it.
Winners take all/most? Not sure I can agree with that. But I’m not sure if this is what you meant though…?
Overall this topic was motivated by the sorry state of bitcoin. Mining is not something most people can take part in. If there’s a 50% mining attack, well, most people will just have to stand back and hope. And the transaction fee market when the network gets busy is a constant pain-point even for regular everyday transactions.
It’d be a shame to see safe network unable to benefit from the parallelism-of-the-crowd (?!). It’d also be a shame if the network had really slow join times, or if relocations took so long it affected the ability to do normal day-to-day operations on the network. I would like to see safe network infrastructure become some sort of taken-for-granted force of nature like oceans or mountains. I’m thinking of the difference between ‘the internet’ (sooooo reliable) vs ‘the web’ (kinda full of drama). Maybe Safe Network apps will have their ups and downs like the web does, but hopefully the Safe Network itself can just melt into the background like the internet does. I feel like achieving ‘sensible’ node sizes could be a big part of achieving this level of reliability and stability.
7 Likes
Not ATM.
What I mean by best performance should win is that traffic shouldn’t be routed to worse-performing mobile vaults merely to spread rewards artificially.
2 Likes
It is more “parallel” as the network grows mind you. Each section is it’s own mini network. I think it’s time to think sideways here. Well I do this a lot when thinking of problems.
We have this balance
- Allow mass join means attackers can get a lot of nodes in the network at one time and easily planned.
- Not allowing fast joins (mass join) means normal folk will see time to join being longer.
So if we take 1 and say, hey no worries, let them do it, node age relocates them all over the shop well before they have authority. So that seems good. But going further they may all become adults and a significant % of the network. Still no authority, but if they vandalise (mass switch off) then it could be a problem, but arguably one we should be able to handle (loss of an Internet subsea cable etc.). So argument for mass join.
If we go against 2 then and say “let them wait”, will they? will this put folk off or will it create an exclusive club where people can boast, I run Safe. (so like Facebook started by not allowing most folk on).
So some arguments against my position (so far).
However then we have split, sections need to split to create new sections, if they don’t then Elders are overwhelmed handling too many Adults. We should know the network never merges, only splits (semi lattice like) Therefore we cannot have mass join - split - mass leave - merge, we will never merge (we can have a section of 1 node though. So that is back to nodes filling up, getting measured on some resource and needing split to create new sections. Maybe I am tired (I know I am 
) but it seems we need to limit the number of nodes per section and maybe the question is limit and measure resource to allow more join or accept any and split really fast based on just numbers of nodes alone? Then the supplementary is should we have a million nodes running when 100 will provide all the resource we require? The rewards are separate, but if folk want to run a node to get paid and it has nothing to do then should they get paid for supplying nothing (worse for forcing the network to do work and give nothing in return?
Then we have churn handling, if we make join almost instant the section has to get consensus on the new nodes, if they join for only a second and stop, then join again, then … it could be an attack, we must mitigate that one. We can of course, we did with Infants, we let them join but cared not a jot about them and almost ignored them, then we relocated them to start storing. So I am just noting that! (clients for instance will join/leave quickly and we do handle that by not getting consensus they join the network, although clients do not form part of the infrastructure. We can consider clients client in a client server model and nodes as p2p nodes that create what the client can consider “a server”, albeit clients do need to understand group consensus/agreement and know a single node cannot be trusted.
Man, not sure I helped @mav it was a brain dump, but perhaps more questions, ideas etc. As you say, interesting conversation and one we need to have. Now I feel we can devote more time to these as the networks in a really good place with the right design now.
14 Likes