Monetary value as data object or projection of ledger entries

No problem, think of it like this. There are say 8 nodes all to agree, 5 of those 8 are enough. When you ask for the data even 3 could be wrong (or the only ones correct, for a few seconds) but the 5 answers you get back are the value at this precise time. So if the data is “changing” it will look like the last value up until 5 of the 8 agree then it’s the new value. The whole group (the remaining ones) then will come to consensus with the 5 already there.

The data chain then enforces this network wide, by that I mean the neighbour sections agree with the value. This is repeated for every value in the network “around” the location that values are stored in. This is the part that gets hard to envisage, but think like this,

Humans in Australia think that John Smith lives at 2 main street Troon, however to really know you speak to somebody close to main street Troon. They tell you exactly who lives there.

The network is like that, to find the truth you go to that part of the network, everyone else may have an idea, but that part knows the answer.

For Immutable Data it does not matter to be close as it’s value is hashed to produce it’s name. So the data would be in fact “John Smith, 2 Main Street, Troon” and the name is the hash of this. But Immutable Data can have changed content so we must visit Troon to see what it really currently is. Not that it cannot change the second we look at it, but in the case of safecoin etc. if you are looking to see if you own it, then it cannot change without you, no matter what the network does (unless it’s hacked completely, like a >50% bitcoin takeover etc.). In this case it’s a mix of network consensus and you owning the private key that ensures when the data belongs to you then it stays that way until you decide otherwise.

[Edit - Maybe the concise way to say this is “The data is changed when a client requests the data. The result it gets is from at least quorum of the group and that is the current correct answer.”

So in short each node changes the data and if more than 50% of them have then the data is changed. Data chains enforces this across more than one group and in fact a bit like a merkle tree, not in structure, but in purpose, will make sure that going fomr a known start point (say the genesis block) you can traverse the tree to ensure the nodes that say this is the data are in fact nodes that can be cryptographic-ally proven to be the correct nodes to attest that this is the data"]

PS: If you think you are not asking clearly, you have met the guy who cannot answer clearly :smiley: but we will get there, honest we will :smiley:

14 Likes

David’s last explanation sounds like what I was thinking here:

In other words, there is not one copy, but several nodes each looking after a copy, who all update their copy when a change has been agreed as valid. So no single node can fake the change or corrupt the value - it will always be out voted by the other nodes looking after that data.

7 Likes

@oetyng I don’t know if you caught this. Each of those nodes has/keeps a copy of the MD data object and the node changes its copy when consensus is met. Data chains ensure a requester gets the latest version of the MD in case some nodes have not yet updated the MD. IE avoiding race conditions etc

5 Likes

To answer the immediate question as directly and quickly as seems possible, there’s a ‘version’ value for each mutable data that’s incremented when it’s changed. This is used (as neo says) like a flag for committing changes and to prevent duplicate simultaneous changes.

See routing mutable_data L370 for where entry_version is incremented when updates happen.

This is actually referred to in a comment below on L429

For updates and deletes, the mutation is performed only if he entry version of the action is higher than the current version of the entry.

Owner is changed by calling clear() then insert(new_owner) - see L571 fn change_owner

Can the atomicity of this mechanism can be broken? A detailed read of the code is required (however is not actually performed in this post, despite the title of the next section).


Explaining By Reading The Code

Now, there’s also the broader issue of how this actually works and why a single node can’t just change the value.

The key word when discussing updates to mutable data is accumulation.

The transaction starts life being signed by the client then broadcast to the network. This is a very ‘raw’ state, not having accumulated any network ‘credibility’ besides that of the client signature. It’s routed to the appropriate section of the network, where it gradually accumulates (or doesn’t accumulate) credibility from nodes in that section. That credibility comes in the form of a signature from each vault in the section saying the transaction is valid according to that vault.

The place in code where this happens in the vault code mutable_data_cache; the whole file is really short and easy to read so there’s no particular methods to highlight.

The decision of each individual vault whether to sign-off on the credibility or not for a particular mutation happens in fn validate_concurrent_mutations

This decision starts life in each vault as a ‘pending write’ - see fn insert_pending_write.

Inserts the given mutation as a pending write. If the mutation doesn’t conflict with any existing pending mutations and is accepted (rejected is false), returns MutationVote to send to the other member of the group. Otherwise, returns None.

The caching of mutabledata by each vault in the section is what keeps any single vault from changing it - all the other vaults in the section would reject any future mutation since it wouldn’t match their cached version. (yes a can of worms has just been opened)

The whole file at vault/data_manager/cache.rs is worth reading to better understand the accumulation process.

The accumulation module is fairly stable and may also be worth reading.

Anyhow, this is some pointers to technical entry-points if that’s the desired approach. It’s not an answer, more a finger pointing in roughly that direction.


Explaining By Analogy

Mutating data is a bit like rolling a big stone down a hill.

A transaction entering the network is considered ‘valid’ if it has the correct signature from the client. This is the first point of validation. It’s like getting the right stone and the right person to the top of the hill. Without that prerequisite, nothing further can happen.

The transaction is routed to the section responsible for it. This is like the person starting to push on the rock to roll it down the hill.

The first vault in the section to be handed the transaction checks if it’s valid both in itself and against their own cached value of the mutabledata object. This is like the rock slowly gathering momentum just off the top of the hill. Once the transaction reaches the section, it begins accumulating to reach quorum (like the stone begins accumulating momentum).

If these checks pass, the vault sends the transaction with its signed approval to the other vaults in the section. Those vaults do the same checks and report their signed approval to all the other vaults in the section. The transaction incrementally accumulates credibility like the stone incrementally accumulates momentum rolling down the hill.

When the transaction reaches quorum, the mutation in cache is ‘saved to disk’ of the vaults responsible for it. Any future vault in the section not caching the new value will be treated as misbehaving. This is like the stone hitting a tree on the way down the hill, reaching the inevitable conclusion of the journey.

Transactions are discrete because they are a ‘single data point’ (ie the broadcasting part) but the event of it being ‘saved’ is not. Despite that, the conclusion is conclusive once reached.

There can be multiple transactions pending accumulation for the same mutabledata at the same time (eg multiple stones simultaneously rolling down the hill aimed at the same tree). The one that reaches quorum first is the one that’s committed.


Explaining By Changing Perspective

Consider some other questions which dig into the nuance of the transaction mechanism (these are equally applicable to understanding the nuances of blockchain transactions)

  • What happens if multiple simultaneous transactions are competing to be committed (ie a race condition happens)?

  • How irreversible are transactions once committed (eg in bitcoin consider zero confirmations and orphan blocks)?

  • How would someone other than the owner spend the coins?

  • How can the consensus mechanism be broken and what is the effect of that (eg in bitcoin 51% vs hardfork are two different attacks on consensus with different effects)?

  • How are new coins created and what prevents them being created illegally?

  • Can coins be deleted or burned and how would this happen?

There are answers to all these questions but they’re too long for this particular post. Use them as new ways of looking into the problem - it might shed some similar light in a different way to the question of ‘how is data committed’.

Best of all (but technically challenging) try to break it. Run a private network and ‘steal’ some mutable data. It very well may be possible; only testing will confirm it.

15 Likes

Great to read, just awesome :smiley:
And, that what you described, yes, it is now clear to me :smiley:

@mav, wow, thanks. Just superb. The clarity with which you perceive and accordingly respond is stunning.

@happybeing, yes this seems to be what actually clears the misconceptions.

Less important things

Something was missing in my picture.
It is really obvious now.

I was thinking, how could this 1 object be securely updated. But of course it is 8 objects being updated, so then the rest comes naturally.(I.e. correct value is the most agreed upon, we actually query 8 sources for the value, and we will get answers from which we can conclude by quorum which is currently true.)
It’s not that I didn’t know that there are 8 copies (and used to be 4). Just, something sent me off 359 degrees… (that means, standing right next to it and just not seeing it) Anyway. :slight_smile:

And regarding nodes getting into inconsistent state, that would only happen if they lost power right after, say for example this line. But it doesn’t matter since if it loses power it will be wiped and restarts, and so the inconsistent state is wiped with all of it’s other data. And it’s a non-existing problem.
(In data chain scenario I do not know what would happen there at reconnect, but I am pretty sure it has been covered.)

However, to run the example app I described, the background issue I started with, which sent me a bit off, is actually not yet solved. Even though the problem was not exactly where I was first looking.

To have a DApp evaluate logic, to have network agreed logic, currently it seems to me a parallel infrastructure would be needed, which if you want to use the SAFE consensus model, would be duplicating a lot of the existing algorithms.

I can write code that makes n instances of an app communicate on the SAFE network, and reach a quorum about a calculation. The manifestation of the quorum having been met is definitely crude, but it’s a start.
But, when an MD is about to change owner as a result of this, then it becomes more tricky.

As long as owners.Len() is not allowed to be > 1 at least :slight_smile:

I will need to read through some more parts to map out what I’m missing. I’ll be back with that.

6 Likes

This functionality is likely to be incorporated at the network level, but some time after the launch. We had a bit of a chat about doing it as an app here:

I say the above because I recall David saying this, and realised it was a much better way to go than building it on top. Unfortunately I don’t have a link to David’s post, but I’m clear he anticipated providing a way to do this using the built in concensus mechanism.

4 Likes

Came across that one before. I was thinking about a smaller scope, single purpose app, and not a computation app.

So just distribution of a particular app’s logic.

Yes, some time after launch it will probably be implemented at network level, but to have a dapp ready for launch some other solution would be needed. Hard to say if solution can be provided with reasonable effort put in, considering that network will have that after some time.
Preferably it would be abstracted away so that it could be switched out for the network implementation later, also hard to say if that can be done in a satisfying manner.

6 Likes

At the current stage of development of the SAFE network, I agree.

This is a post which seems related to your thoughts on smart contracts (replace structured data with mutable data, the concept remains identical):

https://forum.safedev.org/t/multisig-revamp-for-structureddata-appendabledata/135/49

I believe more complex ownership_test functions will be required in the future.

Ownership testing may take the form ownership_test(owners, signatures, script) so the evaluation of ownership is independent of the data object itself.

Also want to add @oetyng I really like the way you’ve approached this topic. Threads like this are why this is such a great community, so thanks for being part of it.

9 Likes