these experts will have a rude awakening when their 22tb nas drive fails and the rebuild takes weeks…
Ahhh, fond memories of my Bubba and Bubba 2, the NAS drive with RAID all setup out of the box, and no feature for rebuilding when a drive failed. 
It was still a nice piece of kit and I kept the units because they look so cool and I’ll put something else in them one day… one day… one day…
3 Likes
For fun this is my childlike understanding of pay once forever.
Autonomi is like the cyclical nature of a river. No?
Every new chunk/record is like a drop in the river.
Adding drops is paid for once.
Drops become fish.
As the river grows more fish appear, they are the nanos, and there are only a certain amount of fish ever to swim in the river.
We catch some of them.
As the river grows there are more fish added.
Those fish will be worth more over time, because there are more fishermen all the time; and after we catch them they are removed from the river, until more are needed later, as if thrown back. And now even more fishermen come to the river to try and catch them.
Since in theory the river may never stop; our nets, or nodes, continue to catch the nano fish, ones which we had caught before; and this is the key, but now those same fish are bigger, a real catch this time, since the river has grown from a stream into the Amazon.
Although we pay once to add more drops into the river, this allows the nano fish to grow, which means fishing makes sense.
Did I get it right? Cheers
Oracles could be added later.
No, nodes would simply not worry about replicating it anymore. Over time it would disappear.
All I’m pointing out here is that solutions can be had if needed. Hopefully it all works as planned.
Free downloads forever is fine for the principle of it but can we avoid repetitive mass download attacks, like a DDoS?
I was already wondering if there are bad actors today already trying to knock out node runners with less robust infrastructure such as consumer routers, asymmetric ISP bandwidth and running on a laptop or pi.
I believe some crypto started with zero cost transactions but then raised it to a tiny but nonzero cost to prevent bad actors from filling the blockchain with useless data.
If the tiny cost of retrieval would be more than compensated by earnings from running a node, it would effectively be free. Why not do that?
1 Like
I hope this can be solved by some kind of rate-limit on gets, e.g. if a node requests the same file too frequently, the close-group responsible for that data somehow block them, but not other nodes seeking that data. Not sure how feasible that is, and a bot-net / coordinated group with thousands of clients trying to get the same data would still be an issue, assuming caching doesn’t avoid it.
Perhaps a good solution would be pay-to-get, but only a small amount when there is excessive demand for a particular chunk. If the network does anything that is free to the user, and no-free to nodes, it’s a potential point for attack / inefficiency in my view, and all of these areas need to have the option of charging for the actions to be taken to avoid abuse.
1 Like
There will be cache nodes for popular content at some point.
3 Likes
Something I’ve been thinking about is that maybe the network code ought to take into account is the mass popularity of some chunks. Imagine the most amazing cat video ever (could be any data but we all love a kitty vid). It’s only a few MB and is stored on a few hundred nodes but they are hit thousands of times an hour 24x7x365 by millions of users because it’s so amazing. There was only 1 payment for that video being uploaded which is fair enough according to the economics. And there is no payment for downloads and that is fair enough according to the economics. But those nodes lucky (or unlucky enough) to have the records are getting murdered permanently. The node runners might see that as a ‘problematic’ node because of bandwidth usage and kill it. Others take its place and the cycle repeats indefinitely. This doesn’t solve the problem and weakens the resilience of the data.
It would be nice if the network watches out for this and makes the records available on more nodes than normal because the records are popular. This is both to increase their resilience because they are clearly important to the world and to increase the bandwidth available for the downloaders.
This is the kind of video I’m talking about (which perfectly describes my job at the moment).
1 Like
Not sure what you are saying here. If you compress a file and its 1TB then it will take 2,000,000 chunks. Compression is not working on the chunks since the file was already compressed to 1TB.
I just assumed you would have compressed it before storing and took the 1TB as 1TB after compression. If you want to say your 1TB is 1GB then of course its not 2 million chunks, but then you are not trying to store 1TB either are you. That is moving the goal posts
Also did you know many files are basically compressed anyhow in their storage techniques. Like images.
Caching is in the pipeline. this spreads the load across many nodes for each chunk.
But also unlike traditional DDOS using a file, the files are chunked and spread across the world. So trying to DDOS using repeating downloads of a file (eg a 100MB=200chunks) results in the download accessing 1000 nodes for the download. This means that where a typical DDOS might involve 20,000 devices repeated downloading of 100MB from one IP address (site) the 20,000 are downloading from 1000 addresses. This is an automatic reduction of 20,000 to 20
So instead of an “easy” bot net of 20K they need many times more to get similar effect. It would be 1000 times if these nodes were cloud, but home you need less. But still it’d be 200,000 botnet at least
Now add in caching just using nodes with those chunks in their inactive list and that 20 might become 5 or 10. But then add in in route caching and its even less.
Good thing this isn’t blockchain and the client does nigh on all the work and there isn’t a blockchain to fill.
And there is mitigations to spam transactions being introduced. So max transaction rate for each computer in the botnet spamming is limited and it does cost real money to buy botnets to do this work. Its not a zero sum game for people using botnets, one reason DDOS attacks last less than 48 hours, most less than 24 hours
EDIT: also remember that the network is handling transactions for every chunk store. So a 1GB file requires 2000 transactions. The network is handling transactions at rates that blockchains cannot even dream of. So what will a expensive bot net of 100,000 computers achieve with say a few transactions per second (doing them in parallel)? Remember the client is doing the work of getting the data and factoring the lag on each piece of the data and the elapsed time for a transaction is not instant.
6 Likes
I like the idea of the node and client being one piece of software - then like with torrents, if you download a chunk, you share a chunk. Not sure if or how that would work on 
, but perhaps even a parallel download client could be created that works as torrent software works.
What’s missing in all of this - current too - is search. Without search, then DDOSing chunks is going to be a waste of time for an attacker - as few will know where it is in the first place.
Pretty sure Maidsafe won’t implement search as there are legal implications. Perhaps the Foundation will give grants for it’s development? IDK, but it’s pretty critical for the long term success of the network IMO … and until then DDOS of chunks isn’t a worry I think.
2 Likes
Isn’t that basically what the tokens are anyway? You can treat them as IOUs for uploads. Iirc, maidsafe realised it was just simpler to represent this as tokens.
I’m not too worried about DDoS tbh. Chunks are spread far and wide and as they are immutable, they can be cached in the hotspots really easily.
There were a bunch of threads about this years back, but basically short term copies of data will swarm to nodes which are being asked for them a lot.
The network will always ensure long term storage of chunks, but that doesnt mean those nodes are the only ones with a copy.
4 Likes
There are still quite popular cryptocurrencies with really zero fees and they are able to fight against spam attacks.
2 Likes
Not following what you mean here.
Yep. It’s a hard problem for attackers on
“Nano”, XNO, is one I believe. Not sure how popular it is anymore.
But it’s not as decentralized and probably not nearly as scalable as .
2 Likes
Xch has free tx, until theres a spam attack, blockchains hey.
2 Likes
I mean, you can run a node which feeds your client with nanos. You could just spend what you have earned.
2 Likes
No didn’t know this…
Sorry for using the term data compression, because its more about learning and adapting.
Really thx 4 taking the time explaining how chunks work.
2 Likes
Just checking in here with some breaking news on how to protect the autonomous Network from masssive DDoS attacks…
The guys at ETH Wyss and Perrig having cooked up Z-Lane, just announced at Usenix… I have read the paper and clearly ETH Wyss and Perrig are on to a progressive solution to be prepared for a DDoS attack, rather than a reactive solution with a known attack surface( which does not involve simple rate limiting…)
It looks as if the ETH Z-Lane tech can be adapted at first glance to be incorporated into the Autonomi Network to keep it up and running, the key is keeping our variant of DNS alive regardless, so that in the event of DDoS being sensed by Autonomi Nodes, Node counter measures can be launched to keep the network alive by re-organizing ie- moving blocks to unaffected close groups…
below is the link to their Z-lane paper which has been proven in a global testbed so its not just an academic exercise.
The use of their EPIC hidden - paths ‘EPIC-HP’ implemented in their routing nodes also figures into deployment of these z lanes
@dirvine If Z-lane or at least the concepts look interesting for us , I know Adrian Perrig and have a direct connection to him based on past interactions with their SCION protocol work (it fixes everything that is wrong with BGP4sec)
9 Likes
Maybe consider to tag dirvine in that post because it seems it could be important. It was mentioned in Dev updates lately that David and some other was working on ddos attacks and such.
5 Likes
This could be very interesting. We are absolutely flooding with work right now to get stable and API complete etc. Please bug me with this when you see a stable network here. I am sorry, it’s just super busy and these things take days to fully consider. I have captured it in my notes as a reminder though
8 Likes