Since this is not designed yet, I’m going to ask about potential ways to do this.
Let’s say I have a group of nodes in an overlay network, communicating via crust.
They have a protocol for reaching consensus on condition to transfer safecoin.
I am wondering a few things here:
If I have a Safecoin, and I want to engage some network agreed logic hosted by this group of nodes, to decide when the condition is met, how would this look like, step by step? Obviously, it is not decided, but I would assume there are some more or less detailed ideas already on how it could be done.
First I have an MD with owner set to me.
a) Then, would I have to transfer the ownership to each in this group?
b) Or could the MD remain mine until the condition is met? In that case, how would we do
Transfer the ownership to the new owner.
So, what I am looking for is basically two things:
Q1. The highest level of abstraction that we are fairly confident will remain stable.
Q2. As much details as possible about the one or two currently most likely solutions.
The reason is that I would like to code up some things that would use the logic, and if I can get some answers to Q1 and Q2 I could for example do a mock and design the applications around it. I imagine that I could do some design progress in the whole decentralized app area by being able to delve into this.
EDIT: gosh… tab enter is not the way to go when writing posts…
so… the rest of the post is coming.
Like for example (just wildly conjuring something up)
var nextUsers = _service.GetNextUsers();
// Create MD ownership rules
using (var mdOwnershipSetH = await MDataOwnershipSet.NewAsync())
{
MDataPermissionSet.SetAsync(mdOwnershipSetH, MDataOwnerAction.AllRequired, nextUsers),
using (var mdOwnershipH = await MDataOwnershipPermissions.NewAsync())
{
using (var appSignPkH = await Crypto.AppPubSignKeyAsync())
{
await MDataPermissions.InsertAsync(mdOwnershipH , appSignPkH, mdOwnershipSetH );
}
// Create the Md with above settings
var mDataInfoH = await MDataInfo.RandomMultisigAsync(15001);
await MData.PutAsync(mDataInfoH, mdOwnershipH , NativeHandle.Zero);
}
}
Consider the group as atomic decision makers. So you send a transfer of a coin and they will agree and do it or disagree. Best consider it atomic.
Not sure what you are looking for here. Consider though any change to MD, never mind owners. We have to ensure that even you logged in from 2 devices and updating that in parallel holds. So you have 2 type 19999 MD updates on a particular address and both go from version 2->3, even there we choose 1 and 1 only and reject the other. So that is similar to you changing ownership.
In terms of mutlsig, there are choices and all based on the valid_successor logic in vaults. So you have an MD with 3 owners, the mutation request will see there are 3 owners, confirm 2 of them have signed the request and if so allow it. atm we have not put that mechanic in place to get the 2 signatures, it can happen out of band at the client side via messages from 1 user to another with a sign request, or the network can cache a request fo X time to wait on the next required sigs. I feel it will be the former though,
Well, I would love to hear you just think loud about it (with no commitment on anything, just exploring the ideas), until you had no more to say That would be awesome, but I can see how it is maybe not possible
All right, so any change governed by the same principles.
Probably the structured data RFC, mutable data went a bit back on that, but only for ease of impl. It will be reintroduced with the same features, but possibly in a different way. The consensus stuff and how we choose 1 is deeper, but there is going to be a video I think. Plus it is evolving even now, this is another full weekend design at my place (17 hour meeting last Saturday, but got a lot covered).
I’m thinking, couldn’t signing the request be implicit by the above?
Then this request is not confirmed by await MData.MutateEntriesAsync(mDataInfoH , mdEntryActionsH ); like normally, but only when, say 2 out of 3, owners all executed the above.
Each of these updates to a field would then not bump the version, until the quorum requested it.
EDIT: Or wait… I think I know what you are saying now. My suggestion above would be roughly this: or the network can cache a request for X time to wait on the next required sigs.
So, instead (as it was more likely to be an out of band sign request), there would be another method, (or optional param).
Something like this:
HashSet<long> _completed = new HashSet<long>();
Dictionary<long, List<string>> _receivedSignatures = new Dictionary<long, List<string>>();
// this is a callback passed to an out of band listener
public void OnSignRequestReceived(SignRequestEventArgs e)
{
await _queuedRequests.EnqueueAsync(e);
}
// this is a blocking process called by some upper layer
// that is running code that requires real time responsiveness
// to decisions on coin transfer, such as a node in a smart contract close group
void Run(CancellationToken token)
{
while(true)
{
if (token.IsCancelled)
break;
var result = await _queuedRequests.TryDequeueAsync();
if (!result.HasValue)
{
await Task.Delay(1000);
continue;
}
var validationResult = await EvaluatePendingRequests(result.Value);
if (validationResult.ShallAgree)
SignPendingRequest(e);
else
// handle this case some way
}
public ValidationResult EvaluatePendingRequests(SignRequestEventArgs e)
{
if (_completed.Contains(e.OperationId))
return ValidationResult.ShallNotAgree;
if (!_receivedSignatures.ContainsKey(e.OperationId))
_receivedSignatures[e.OperationId] = new List<string>();
if (!_receivedSignatures[e.OperationId].Contains(e.Signature))
_receivedSignatures[e.OperationId].Add(e.Signature);
// some additional logic, we can't just agree to any incoming request..
return ValidationResult.ShallAgree;
}
public void SignPendingRequest(SignRequestEventArgs e)
{
var (mySerializedMdHandle, mdVersion) = await _wallet.GetRandomCoin();
using (var mDataInfoH = await MDataInfo.DeserialiseAsync(mySerializedMdHandle))
{
if (!await MDataInfo.IsQuorum(mDataInfoH, _receivedSignatures[e.OperationId])
{
await Swarm(e); // out of band message to other owners
return;
}
using (var mdEntryActionsH = await MDataEntryActions.NewAsync())
{
// N.B: adding signatures as last argument!
await MDataEntryActions.UpdateAsync(mdEntryActionsH, "OWNERS".ToUtfBytes(), newOwners, mdVersion + 1, _receivedSignatures[e.OperationId]);
await MData.MutateEntriesAsync(mDataInfoH , mdEntryActionsH );
_receivedSignatures.Remove(e.OperationId);
_completed.Add(e.OperationId);
}
}
}
As for initiating a transfer, I’m thinking like, one level above this I would possibly do so:
// I have a coin which I own with 5 others.
var coinHandler = _factory.GetCoinHandler();
var recipientOwners = new List<string>{ ownerD, ownerE, ownerF }; // say I set 3 new owners here
await coinHandler.SignTransfer(amount, recipientOwners);
// I am the first to sign, so I request signature from all other owners,
// they in turn swarm the request. When any owner has a quorum of requests,
// the ownership transfer is requested and the signatures sent as parameter.
// next time this coin is to be transferred, 2 out of these 3 new owners need to do the above
so I imagine the quorum rules are already set on a coin when doing this, so I would not pass in any info on that.
Sounds like a good weekend. You’re doing such a great job (almost sounds flat to say it). I’m guessing you will be having more of those then.
Looking forward to the video, you mean like whiteboard presentation by you/others or animated CEP stuff?
For safecoin 100% so, for other types it is more debatable as the network has to do something, but that will be a debate I think. Personally I feel it is not adding data really so should always be free, but we need to detail it fully.
When we have multi-ownership of an MD, (so regardless of type)
and let’s say we will change ownership with 1 single request (as per the example above), which must contain a quorum of current owner signatures for the ownership change to be valid…
So, I’m thinking here that if we have a group of owners, that are to agree on the owner change, there will always be one of the owners paying the Safecoin cost of updating this MD (that is, if the ownership change will cost. But the alternative - no cost - seems impossible to me… it is 100% certain to be spammed to annihilation.)
When only one of the owners pay for the ownership change, we could have a problem if we have constructed a close group, that is rewarded for reaching agreement on logic (i.e. network agreed logic - NAL): one of the nodes will be paying for the update, while all partake in the reward.
Question: Has there been any discussion on how costs for ownership change is distributed among previous owners? (i.e. in the case choice of design would fall upon having costs for it. I know it was said that it is not decided whether there will be a cost or not)
Let’s say the cost would only be paid by the final updater, then we have a problem to solve for the domain logic in the application layer. Who will pay, and why that account/node? Designing around this will be an added complication to application design. The simplest would be if all owners paid an equal share. But then again… if any of them do not have balance, can the ownership change not go through? Would it be possible to set rules for covering for insolvent owners? Aah… the abyss of complexity opens up…
Taking a step back, looking again at the NAL execution:
A possible design solution for such a network agreed logic group (NALG), would be that only the node finally doing the update with the signatures, will be reaping the reward. This creates a lottery effect, and competition between the nodes in the NALG, since they all want to be the first one to have all signatures.
I am wondering though: Would they try to play the other nodes somehow, in a way that would be detrimental to the goal of efficient NAL execution? Maybe waiting with swarming their signature until they have quorum-1 signatures. But if all do this, then there is no swarming at all, and there will be no rewards for anyone. Seems like if it is gameable, it would be quite advanced to do it, especially if all nodes tried to game all others. The interesting question, I guess, would be: Would it ever be beneficial not to swarm the signature as fast as possible?
Spammers can be punished / rate limited so transactions can be free-of-safecoin-cost but not free-of-consequence.
Punishments is something that’s extremely powerful and unique to the safe network. This alternative mechanism to ‘purely reward based’ has gone completely unnoticed and unappreciated. I’ve been meaning to write about it and the benefits it brings, but for lack of time etc.
I think (like bitcoin) so long as the transaction is valid when it’s broadcast (ie has some fee included if that is needed to satisfy validity) then it’s up to the users creating the transaction to decide who among them pays the fee.
This assumes the transaction is signed one-by-one. It may be that the transaction must be fully signed before being broadcast or it’s rejected by the network. Sounds reasonable to me, although I do concede there are benefits (and drawbacks) to signing one at a time. You may find BIP-174 Partially Signed Bitcoin Transaction Format interesting since it also deals with this one-signature-at-a-time problem.
On a more general level of updating MDs (ie transacting safecoin), have a look at how pay-to-script-hash (P2SH) transactions work with bitcoin. This literally means there’s a script of arbitrary complexity forming the basis of the transaction.
This gives a lot of power. You can create a script (as in a programming script) and it might involve checking signatures or block heights etc. This script forms one part of the required information when sending funds. The other part is providing the correct inputs (ie signatures) to that script to make it return ‘true’. So transactions in bitcoin can be quite complex without that complexity needing to reside in the parts of the transaction that end up living on the blockchain.
To further generalize, transactions are rejected or accepted as valid on the rule transaction_function(inputs) == true. The most common case is check_signature(signature, pubkey).
So for safe network, multisig isn’t (or shouldn’t be) ‘a set of pubkeys with >0.5 valid signatures’. That’s how it’s described in deprecated structured data, but to me this is the wrong way to go.
It should be ‘pointer to an immutable data which is evaluated with the supplied inputs’. That immutable data contains a script which is evaluated by the group for validity with the supplied inputs, but importantly the script is outside the md itself so it removes the restriction of size and complexity inherent to any single md.
This allows for arbitrary complexity of transaction conditions (maybe restricted to a non-turing-complete language so it has deterministic execution and stopping constraints). You might have a transaction that says ‘if immutable data XYZ exists on the network, and a signature for that data using pubkey RST is supplied, change the owner of the md to ABC’. This is a way to release funds when a document is published and signed. Can’t do that with ‘majority of signatures’ multisig
Multisig is simply a ‘basic’ scripted transaction (or if you’re ethereum ‘smart contract’). It’s better to move away from the multisig concept and abstract one step further to ‘script execution’.
Single signature matches? It’s a basic script.
Majority of signatures match? It’s a basic script.
Complex signature weighting conditions? It’s a script.
Non signature inputs? It’s a script.
Waiting for a future event? It’s a script.
Transactions are scripts. They don’t need to be arbitrarily complex straight away, they can just be simple and highly constrained scripts, but eventually, all transactions become arbitrarily complex scripts.
I know this doesn’t directly help or answer any of your questions! But I needed to get it off my chest. Maybe it helps with your abstractions.
I thought this would be the case. Saves on the section retaining state information about the update waiting for all the signatures needed which might take days. Imagine a thousand such MDs in a section. (large database being massively updated)
Also if each sent a signed request (one by one) then I’d say each pays their one “PUT” cost for that. If a request with all signatures is made then the one actually making the request would pay the one “PUT” cost.
Good point.
So we could consider owner changes free of cost. Even so… I go back to this:
… and if we look at an owner change as just mutating a field on an MD, then it would be introducing a special case to make it free to update a specific field.
Even though owner field is maybe most often not very large, it is by no means impossible to imagine a case when most of the MD size is actually comprised of the owner field. Some giant crowd ownership stuff.
And then it becomes so much more obvious that changing ownership is just another field mutation - and is it really good to introduce special cases? I think most agree that they should be avoided as far and as much as possible. But maybe there is a good reason for a special case here.
Yes, so if there is a cost with owner change, and also if we consider any other MD mutation, which will have a cost: If we are using an overlay network of nodes that are supposed to secure a distributed logic execution by group formation rules and consensus etc., then there will be 1 single node in the group that has to carry out the update, and pay the PUT cost to SAFENetwork.
And this is what I was referring to, here:
And that would equal to the users creating the transaction, have decided that the rule is that the first to have all signatures, and successfully updates the MD, pays the PUT fee and reaps the reward.
We might be talking about different things here, because I am talking about agents, executing code on behalf of clients, as to enable decentralized applications. So that’s where “reaping rewards” come in. Clients pay agents to execute the code. A smart contract execution is fueled by gas put in.
However, what you say there, I can not readily process it into some concrete example, so I’m actually not sure I know how you are imagining including a fee within SAFENetwork context.
What I am designing is actually using both of these things. There is code executed, as to validate any state change, and when consensus is reached on the outcome, the state change is carried out (MD update(s) and/or ImD creation(s)). However, the signatures and pubkeys are required for a node to decide whether there is a consensus: it knows its group, and it has to verify the signatures. Additionally, the hints from David above, was that we would pass in valid signatures when doing the SAFENetwork operation, so the node needs to collect them. And so that’s how these things are tied together in what I am working with.
I agree. The only thing that differs is that we are talking about slightly different technology/implementation. What I do not understand is how you mean that a group evaluates validity, and reaches consensus, without the use of signatures?
I mean, how does the above replace the use of mutlisig?
I feel there are some big blank spots in our common picture where many misinterpretations dwell.
Actually, the way I describe it, it is done too. It’s maybe not the best way, but basically, nodes are either notified of the document by the publisher, and evaluate it, and then release funds when they agree, or they actively keep querying for the document at some expected location, at some interval, and then when found they agree on the validity and release funds.
That’s what I meant, like in the code example, the wording was not good above I see. So, the signatures are swarmed and with final updater, I meant the one that is finally doing the update, as it has all valid signatures. This is the one that will pay the cost. And I agree it seems reasonable.
The problem arising is that although someone must do the update as to receive the reward, no one wants to pay the put cost. But then again, if they all agree that the one paying the put cost also reaps the reward, then they will rush to be the one doing the update, and paying the PUT cost. I actually think it could be a good rule. The only hesitation I had was if it was somehow gameable, in that every node wants to be the first one to have all necessary signatures. But I do not see a clear way how to actually achieve that by not sending your own signature as soon as you can (we would of course want all nodes to perform tasks immediately, and not stall and wait for beneficial opportunities).
So, to summarize:
Multisig as I see it is the SAFENetwork implementation for managing multi ownership operations on data.
Code evaluation (scripts, etc) is what the agents are using as to decide basically anything.
When they have agreed (they know that they agree by having signed the output result of the code execution), they provide their signatures over the desired state change, which is then passed to SAFENetwork.
If you request of a section to do any update (even adding a signature for a multi-sig update) then they are charged. Its work the section has to do and so the user gets charged for it.
So the better way is for one user to collect the signatures then make one request. Only one bit of work to be done and only one “PUT” cost
Mm, that’s how I imagined it at first, but by this…
… I take it as it will be a single request with all necessary signatures.
This would be the benefit of it. And I like it because it doesn’t require any application layer logic to coordinate it. It is already built in that every consenting owner in a quorum pays their share of the update. Then any implementation that sees the need or want of some other distribution, could implement it.
But I would guess that many many cases would be wanting just an equal split with no extra complexity and logics fuzz (so, enforced by SAFENetwork would simplify it).
Which is why I entertain the notion that either is possible and supply my thoughts on charging.
the collection of signatures into one request is easier on the network and would cost less to the users collectively. But allowing the network to accept signatures piecemeal is possible but adds problems like
a second update is required before the first has collected enough signatures. Processing headaches for the APP and network
the signature sent would have to be stored with/in the MD to say what it is for, network event timeing, etc. Network event timing is an issue too since sections evolve and MDs can be moved to child sections and network event time can be a different start value.
what if the last signature never arrives. How does the APP tell the user it failed? Seeing as it could be 12 hours later and the user has gone on holidays. Better for the app to collect all needed first so then the update can be done quickly and any errors reported in a timely manner.
Consider for a moment the seemingly unrelated case of clients storing data. A vault storing the PUT data doesn’t earn the fee the client pays to PUT data. But there is an ‘indirect’ benefit to the vault because clients doing a PUT involves recycling safecoin so the rewards for GET become a bit more likely due to slightly increased supply. So the vault benefits from PUT, but not directly because of the fee.
So maybe in a similar way, agents are not directly paid the fee for the mutation, but the fee goes to a ‘pool of rewards’ which the agent can access via some less racey mechanism. I think this is how it works, since a mutation is a PUT, but anyhow the idea is what I’m getting at, not the specific implementation.
It may be that a client wants ‘no condition’ in order to mutate their data. This requires no public keys. See Script - Bitcoin Wiki especially 2.4 Anyone Can Spend and 2.5 Freezing Funds Until A Time In The Future. There are cases where mutation does not depend on client permission (ie signatures).
It’s true that signatures will almost always be part of the mutation. But it’s incorrect to assume they will always require signatures.
To clarify, the group itself and the messages between the group must use each others signatures for secure messaging and consensus formation, but the inputs to the mutation do not necessarily need to include client signatures in order to mutate data (but usually will).
That would be awesome, but I can see how it is maybe not possible 
Who will pay, and why that account/node? Designing around this will be an added complication to application design. The simplest would be if all owners paid an equal share. But then again… if any of them do not have balance, can the ownership change not go through? Would it be possible to set rules for covering for insolvent owners? Aah… the abyss of complexity opens up…
