A path to decentralized computing on SAFE

Apps
1

Just doing a little brainstorm here. Feel free to change, add, remove to the ideas purposed below.

That most computers don’t use their CPU that much isn’t new. That we, as people at home have a lot of computation to offer isn’t new as well. Folding@home is a network of 167000 computers outputting 36000 teraflops of computing for studying Alzheimer’s, Huntington’s, Parkinson’s, and many cancers. Could we do decentralized computing on SAFE? Yes, we actually do a little of that while running the Vaults and Clients. Calculate how your close nodes see the network is a little part that, just like calculating and verifying all digital signatures and Chunk-hashes. But what if we want to do a little more? What if we want to offer a market to Apps and Nodes on the network where everyone can tap into the CPU power of others on the network, in a Safe and decentralized way? How do we determine who’s doing what? Who’s paying who?

XOR again
Because we already are in XOR-space, maybe we could use that to our advance. And because we already have this thing called Safecoin, maybe we could use that as well. But who’s gonna do the computation? This is where XOR comes in again. We’re already in groups of 32 nodes, so that these calculations should take place in there is quite obvious. But do we want our close nodes to do these computation with us? That way we’re limited to just 32 nodes. What if we have a very big amount op computation we want to get done? We have to find a way where different groups do the work. What I propose here is a new type op PUT. Maybe we should rename it to doing a COMP. We have a lot of computation we want to get done, and we want different groups of nodes to do it for us and pay them.

COMP request
Let’s say we want to compute this little script below.

{400+400+400+400}

How do we know which group of nodes should do the computation? Well, let’s compute the SHA512 Hash of this computation and see what comes out. The outcome is:

"694138866B0098C1DF2807D9A2190421E48B99CF874DB16BB68D9698B39E12E4C78EF0ACB6EE50B5C241EC6BE8FAD09E7970DF7767C87A5010BB6F6A6B8B4C03"

And remember, no matter how big we make the block that needs to be calculated, the SHA512 will never get any longer than that.

Who’s close?
Now we have a computational block with instructions, and we know the SHA512 hash, we actually can reach out to (send COMP-request) the closest group of the hash to determine if they will take our computation. So we send out the request and the group (nodes) that are closest to the hash maybe responds that they have capacity to do calculations. So they message us back and request a payment in Safecoin to do the work. We, pay our Safecoin and sent them the block with instructions (all signed by our close nodes after they’ve verified we did the payment). The next step is of course, for the receiving group to do the actual computation. But who’s gonna do what? Do we have 32-nodes doing the same computation?

2 out of 3
It’s the responsibility of the group close tot the hash of the computation to actually perform the computation. And the 3 close nodes to the hash (of the group) will the offered the chance to earn some Safecoin. If 1 or more nodes are already busy (say their CPU is to busy) it can message the other nodes in their group that they’ll pass this time so the next one close in line can take over. In this way, the group of 32 takes responsibility for doing the computation and makes sure it get’s done. So even while only 3 nodes did the actual computation, the whole group will sign the outcome and send it back to the node who did the COMP request. The answer will contain the hash of the computation block (it’s won’t send the computation block back) and the answer +hash. it could look something like this:

"694138866B0098C1DF2807D9A2190421E48B99CF874DB16BB68D9698B39E12E4C78EF0ACB6EE50B5C241EC6BE8FAD09E7970DF7767C87A5010BB6F6A6B8B4C03"
{1600} 
"A3B851A4F07F0D5B00988701FB7A2B4EC174FF071A7B3F7A602194FA943A03086BDA7079576E2EC6A291B4869623073D3AD9E5AA9D509C567DBFE4F5B869D1F1"

And here it is. Decentralized computing on SAFE using XOR-distance to route computation over the network. Payment is done in Safecoin, computation by 3 nodes (answer should be 2 out of 3 protected).

Pros:

  • Can be implemented on XOR-infrastructure. Uses routing which nodes are familiar with.
  • 2 out of 3 and a group that signs makes a whole group responsible. Outcome should be correct.
  • When one node isn’t able to compute, within the same group a new another one is found.
  • The group can reject a request when’t it’s already to busy, this signed message gives the next close group the opportunity.
  • A group can take more computation blocks at once, because only 3 of the 32 nodes will work on 1 block.
  • If due to an error, a group got payed but wasn’t able to route the answer back, only a little Safecoin is lost.
  • PKI is used between the sender and receiver of the COMP request. So only the group of 32 nodes (yours and group close to hash) will see the computations that needs to be done, even while the computation block itself is not encrypted.

Cons:

  • Only as fast as the slowest of the 3 nodes. No incentive for nodes to provide more computation.
  • Not all nodes want to do computation. So maybe a specific XOR-layer is needed for people who do want to.
  • It will cause extra blocks/traffic in the network. That will make the routing of the common Chunks slower.

Let me know what you think. I’m quite interested to hear if this could be done and if there are different approaches to decentralized computation on SAFE.

17 Likes
2

Curious if this is the best way to do it, but sounds like a proper method. Probably for safety I would do a 3 out 4 in the beginning, but maybe you can switch to 2 out of 3 when more trust is obtained or even switch to 1 out of 1 when you fully decide to trust a specific node (maybe build in some other after the fact methods to check if the calculation is valid, but this depends a bit on the specific calculation, but can be done for very large projects).

1 Like
3

Yeah, 3 out of 4 is more safe. We can even go higher. Problem with this way of doing things is that there’s no incentive for people to buy faster computers and provide more computation to the network. But I don’t have a clear idea on how to fix that problem. Maybe 32-nodes should challenge each other with a little computational request. That way the close groups knows how much it can do. Another problem occurs when people get a computation that can’t be done. Like calculating the hash for a Bitcoin-block. On a CPU-basis this would take years probably. So a time limit should be built in as well.

4

I think rank can be an important parameter here. Low rank->higher group size. Once you establish a good rank, you can rent out yourself for larger projects and only do the calculation yourself (provided that the party who wants to carry out the calculation can do an after calculation check if the calculation was valid). If you keep doing good work, you will get a higher rank and can earn more safecoin that way (sort of reputation system).

Other method is to make a rank based on computation speed so that you can make groups of 3 or 4 computers which all have a pretty similar CPU power.

1 Like
5

An alternative to using group consensus to ensure correct computation us to use zkSNARK.

In this case only one node needs to do the work, while the group would validate the result.

The group would select one or more willing nodes to have a go, but the first to complete would win, so it would pay to be a faster compute node.

The group verifies the result (this is a property of zkSNARK), pays the node, and returns the result.

8 Likes
6

What abiut 90% for the winner and the rest split between the losers. This winner takes it all attitude is flawed I mean weren’t the losers trying? At least there should be an option for “no thanks, I don’t want to even try and waste my cycles while this petaflop monster is in my group” :smile:

3 Likes
7

Bare in mind that ethereum runs the same code on every full node on the network (https://forum.ethereum.org/discussion/2380/in-ethereum-when-a-smart-contract-runs-does-it-run-on-all-nodes-how-is-concensus-achieved). Even if safe net only managed 32 nodes running the same code, it is a massive improvement.

Ofc, we should be aiming for as much efficiency as possible and if a other nodes can easily confirm that a primary node completed the code properly, then even better.

1 Like
8

The details are all up for debate, but yes, I think participation should be optional, or at least none participation not penalise you.

The group composition will be different for every calculation - the closest to a new point randomly distributed around a particular node - so its pretty unlikely you’ll have the same two nodes in the same group for different calculations. At least only rarely.

9

I think it is not the best option to go for a blockchain type of race condition on earning. After all we are not looking for a raw computing power as a security measure. @happybeing already alluded to mechanisms for client verification of calculations (zkSNARK), or alternatively 3/4 (or 29/32) redundant calculations are both very good candidates. I believe the reward should primarily be tied to a correct computation that is independently verified. If verified to be correct, odds of obtaining the award should be high. There is already a push for better computational effort as faster calulcation allows you to gain more awards.

We can focus the framework on big calculations, as no-one will pay a network to perform a simple calculation their own CPU can do in 100ms (or a few minutes). HPX compile the full calculation in blocks of futures, ie blocks of calculations that can be executed as soon as all needed input result are calculated.

Such as system would give a tree structure of sub-results, which maybe block-tree wise each participating node can write its name in. Full completion releases the escrowed award among the nodes.

comments?

6 Likes
10

As a non-techie, I was wondering about how this big data computation stuff is all achieved without creating a similar thing to the Bitcoin Asic mining race which leads to centralisation - as @BenMS says:

I’m a bit confused as to how the “problem” identified by @anon40790172, is in fact a problem? Is it not rather a benefit, or if not why not? Just not grasping this bit… :smiley:

11

What I would like to see is the ability to continously run a decentralized service, as real-time as possible.

Just like we can now create our own types of StructuredData, I would like to be able to design my own type of vault persona, one specifically to run my service on SAFE. Running such a persona would naturally be optional for vaults. Those vaults that choose to run this persona would form their own mini DHT on top of SAFE’s base DHT (using the identities of the base DHT). A client that wishes to use the service should be able to connect to a close group on this mini DHT. This close group is then responsible for handling the client’s interaction with the service.

It might be possible to create a standard reward algorithm for running such persona’s, based on the amount of resources used. Alternatively, the reward mechanism could be part of the custom persona design.

Actually, I have a question. What is the difference between smart contracts and decentralized computing? :blush: It seems to me the smart contract is a particular method to achieve decentralized computing?

1 Like
12

Let me clarify, if you have a faster internet connection, it will help you as a Vault. Because you can deliver a requested Chunk a bit faster, giving you a bigger chance to farm Safecoin. Same for more memory, you can add extra Vaults if you provide more GB’s so bigger chance to farm Safecoin again. But in my case, with computing? The network has no clue about how fast a node can do a computation. So when I’m in a group, providing computation, say doing computation as 1 of the 4 nodes who’s doing computation, why would I buy a faster CPU? The computation is only as fast as the slowest of the 4 nodes. If the network would have a way to understand how fast people’s CPU’s are, than maybe we can find a way where the faster nodes can do more. But without that, no need to buy a faster CPU.

3 Likes
13

Was thinking about it myself. Big difference with decentralized computing is that you don’t share the result with the whole network as you do in smart contracts.

1 Like
14

Aaah… I get you, the Network itself has no way of knowing cpu power. I was more considering about the way the system/algorithms are optimised to reward smaller farmers in regard to storage space to give access to earnings to third world etc. Do we not in some paradoxical way need to optimise things to favour more lower power cpus? :smiley:

1 Like
15

Isn’t it as simple as a dapp that participating groups report to. You have opt-in in your local group then the group is given some standardised math test that clocks the speed of a fastest node which is delegated as comp node of the group (you can even delegate within the group a node that responsible for reporting, fastest in terms of ping to dapp) the rest of the group is there to verify. The dapp params like min comp speed etc. can be tweaked by the user… :smiley:

1 Like
16

Reading your discussions I daydream that down the line, after SAFE has become the world hard drive, then it is ready to become the world computer :smile:

Reminds me of Asimov short text The Last Question

2 Likes
17

Yeas, probably some things like that will be build. But there may be some nodes in a group of 32 that don’t want to have anything at all to do with computation. Think of mobile phones, etc. So they even don’t want to confirm/reach consensus to any computation done in the group. Maybe an App can go around this, form computation groups using an App.

18

@BenMS

We can focus the framework on big calculations, as no-one will pay a network to perform a simple calculation their own CPU can do in 100ms (or a few minutes).

Don’t forget simple calculations on large data sets (finite element analysis, big data processing, search etc). Where the problem can be split up into very large numbers of simple calculations. This is a very suitable application of distributed computation.

3 Likes
19

Yes, that is what Im hinting at, the reward system can be built on the assumption that the reward for the full big calculation can be split up in small blocks

2 Likes
21

What happens if I do not return 1600 on this but another value?