DOS Attack caused by Data Overload

When minds come together and ego is left behind, magic happens. I think this is what we just witnessed. MaidSafe has been like this since day 1, it really motivates us in the office when this magic happens. We always leave a room and nobody cares who thinks of these ideas, they immediately belong to us all. This community seem to get that very fast. It is very powerful. It drives me for sure. Like a refuel :smile:

It is an amazing project we are all involved in. This solves a huge amount if edge cases for sure. Phenomenal work everyone.

Interesting. Would there not be a problem with people using sound processing software to tweak their voice to create a new ID? If it is possible, then it will get scripted, IMO.

FWIW, I still think that a market solution is more appropriate here. People can just pay for what they use, beyond a network agreed charity base storage amount. As they can also earn Safecoin for providing storage, they can offset the cost this way too.

This would also have a nice side affect of stabilising Safecoin demand.

Somewhat off topicā€¦
I speak only for myself. These are hard questions Iā€™m trying to answer.

One member wisely said, ā€œdemand is insatiable.ā€ This is the main reason I favored, ā€œyou can only get out what you put in.ā€ Realistically, if a person already has a smart phone. Iā€™m pretty sure they can afford a hard drive. It cost much less than a smart phone. If they canā€™t afford a hard drive, what makes us think they can afford safecoins?

Using safecoin to buy storage could break a 1:1 relationship (resource for resource), because safecoin would be traded on an exchange. This means speculators can drive up the ā€œnational currencyā€ value as well as crash it. I suspect it will fluctuate like Bitcoin.

So how do we give resources to people who cannot afford it? Charity. I believe this and mass adoption is the reason we decided to make the SAFE Network absolutely free. The problem is, free is often abused. If we do not believe this is true, then why are we trying to make sure people donā€™t make multiple accounts?

If we can prevent users from making multiple accounts and we are providing charity resources. I would like to call it, Network Allowance.

Network Allowance = TBD by people smarter than I. It must also scale with the network.

Total Upload Limit Per User = (Total POR contributed + Network Allowance)

Iā€™m still on the fence. Maybe the community will provide more storage than it uses and this precaution will be irrelevant. Anyone got a crystal ball?

Yes, the market based option would be very stable, but you couldnā€™t provide a ā€˜charityā€™ free part; the question is how do you prevent millions of accounts being made by a single attacker.

This is one of the issues with all of these systems for sure. If we can have the close nodes perform some tasks to ensure this is not scripted. That part is more difficult for sure.

Two things.

It seems like if you give 500gb of storage, then you are alotted 500gb on the Safe net. Then you gain or lose based on your ratio of sharing + demand.

Second, maybe the cost of space is variable? Could the network determine demand? Like how Lyft/Uber has surge pricing. If people start requesting too much space, then the rewards go up to compensate people and incentivize people to share. Then if thereā€™s too much space available, the network lowers the payouts. Maybe thereā€™s a way for the safe network to hold reserves of safecoins that it can dole out or hoard in these situations. Is that possible?

We can allow the network to set the charity rate - just ask the farmers how much storage they want to give as charity. We want a distributed decision in a distributed system, so why not ask those who are providing the resources?

The vault could be configured to allow an amount or a percentage. This could either be accepted on a per node basis or an average storage size could be defined based on all vaults (more like a vote, based on personal stake in GB etc).

In short, the people who provide the resources then get to agree how much they can spare.

1 Like

Re space available, Iā€™m sure we can get an approximate vault utilisation figure, which would dictate the payment amount to farmers.

Higher utilisation => higher payment.
Lower utilisation => lower payment.

This payment amount could then be reflected in the storage fees to get the best, sustainable, price for users.

Yes, I think the current mechanism covers this. Rate of mine attempts is regulated by stored space, the actual attempt made on a get of some useful information (based on the rate set).

I like the idea and control over their own machines should certainly be in the hands of the farmers. The only problem I see, is that the ā€˜charityā€™ part (I really donā€™t like to call it that actually), is not so much the charity part as the vulnerable part to attacks

Technically, it is not charity on the giverā€™s side because the Network pays 100% for all resources being used. But it is charity on the receiverā€™s side because they are getting a small portion of those resources for free. Thus I called it an allowance which everyone can use, even the givers.

Another hard questionā€¦ will givers unknowingly max out their upload limit anyway? I have to reconsider my original formula because it might be self-defeating. If we give farmers the ability to allocate resources for the network allowance or charity, whatever you want to call it, then we can do something like this?

Allowance Per User = Total Allowance Available / Non-Farms
Iā€™m assuming non-farm or ā€œclient modesā€ will be mostly mobile devices and users with no resources

If Total POR Contributed > Allowance
Then Total Upload Limit = Total POR Contributed

If Total POR Contributed < Allowance
Then Total Upload Limit = Allowance

Feel free to alter or improve my formula. Just trying to make sure those who need the resources get them. Also farmer resources not allocated to the allowance is protected/reserved for them to use. So an attacker cannot take it away from them.

1 Like

I like the use of upload limit as we can ignore deletes and keep the maintenance levels very small. To set a limit of network average would be relatively simple and easy to calculate. I like the approach. WE have the link between vaults and uses in place just now, although commented out for the testnets, but this is also fairly easy to establish to, so no huge code changes.

This seems like it may be en route to a neat solution @dyamanaka and @benjaminbollen , what do you think @Traktion it seems close to your thinking as well, although not quite asking for donations from farmers (Who would earn more if they did donate, perhaps we can calculate that actually).

We still have the huge number of account situation though as @benjaminbollen has pointed out. Maybe that should be treated as a separate issue ?

Iā€™m in favor of whatever works in the most efficient way possible. We are making progress!

1 Like

It doesnā€™t solve the issue of multiple accounts, but with regards to limiting the free usage:

I feel for a(n upload) limit on private storage. Public storage should be free and unlimited.

To prevent abuse of the unlimited free public data the network can calculate the entropy of the public data. If the entropy is (close to) maximal, the public data is either encrypted (for private use but trying to bypass limits) or pure/pseudo random data, meant as an attack on the network. In both cases, when a threshold for a single account is passed, the account can be blacklisted.

I am assuming that public storage is linked to a public account (MPID). However, this can not be linked to a MAID account, presumably? That is an important hurdle for the proposed scheme above :smile:

This may be a little tricky, but could work. At the moment all data goes through the Maid accounts, everything is encrypted, except directories (holders) for public data. Soā€¦

We could have public data, only uploaded by MPID idā€™s and they first provide a directory with data maps. The MPID managers could allow the chunks to be stored that are reflected in the directory, free of any charges. This would be a total separation of private and public data and may be a good thing actually.

In this way all data stored by MAID clients is considered private and all data by MPIDS is known public.

All chunks etc. are fully encrypted in any case, but the directories are the parts we can see in public data, which makes this possible.

1 Like

Itā€™s possible I actually misunderstand the role of the different account IDs.

I assume for publishing webpages, applications and public knowledge data this is done by some ID. After rereading Vault wiki I think M Share ID is responsible for sharing it in a group of IDs. Could this also be done for open/public data as described above with a MSID?

I assume the name of a MSID would function as a sort of DNS service. Once itā€™s registered, it is taken. Here I do strongly encourage fixing a price in safecoin/month for ā€˜rentingā€™ a MSID name. Otherwise we will have the same problem again with current domain names: useless companies that hord names and sell them at outrageous prices. The garbage collection of the network should also recycle old/underused names.

Current domain names are already rented rather than free, and it is impossible to set a flat rate fee that is both fair (not punitive to ordinary folk) and will prevent land grabs of desirable properties.

This needs a ā€œproblem/solutionā€ thread!

Perhaps the network can make it hard to trade in public IDs? Thatā€™s how facebook tried to combat this. The other way was to require a property to gain reputation (likes) before it could apply for a vanity name, after which the name could not be changed. These are quite restrictive measures, but would probably be effective of they could be enforced.

I was thinking that the farmer wouldnā€™t get paid for what they give as charity, just like any other charitable giving. That is, they are only paid for what they sell, not what they agree to give away .

If the free space was always be divided equally between users, the worst an attack could do is decrease the free space available to others. As you could always keep dividing up this charitable amount, you would never really run out of free space; users would just get less for free and would have to pay for more.

Ofc, that doesnā€™t stop bots creating users and consuming space, but this is the problem with giving away free stuff - people often abuse common resources. At least their abuse would be limited to the charity space, rather than all space though. If we can detect bots, maybe that abuse could be decreased further too?

Edit: typo

A network average is better than no cap, but it may be over generous. This would result in those paying having to heavily subsidise those who are not. This may cause the whole network to become uneconomical (as people would buy cheaper storage elsewhere).

Imo, this is why it is important to let the farmers set the charitable storage amount. It can then be adjusted by the farmers to suit the economic climate for storage:

If too many people are free loading => charitable storage may decrease.

If few people are free loading => charitable storage may increase.

I am sure the network would find a rate which both pleased farmers and users. There is an incentive for farmers to offer free storage to ā€˜hookā€™ users into the system. They may later become users who increase demand for farmers after all.