This post to inform you of a change in SD deletion implementation that took place one month ago, that might be of interest to developers and that seems problematic to me.
Previously, deletion was equivalent to posting a new version of SD with an empty payload. The advantage of doing so was that history of a SD could not be rewritten:
Only the owner of deleted SD could recreate a SD with the same name
If the owner wanted to recreate the SD, he or she had to start from previous version number + 1
Now deletion of SD is a real deletion, meaning that it is removed from all chunk stores in the network. As a consequence, anybody can recreate a SD with the same name and restarts from version 0.
Previous functionality can still be retrieved by explicitly posting new version of SD with an empty payload. So now there is a choice on how to delete a SD:
Issue a DELETE command when we don’t bother that someone else recreates the SD
Issue a POST command with an empty payload when we want to be sure that nobody recreates the SD
The regression comes from the mere fact that real deletion is now possible: Previously an SD owner could claim that the content of a SD has not changed just because its version remained the same. Now this claim cannot be made anymore: To modify a SD without modifying the version a bad actor can delete the SD, put the SD with a new content and repeatedly post the new SD until it reaches again the previous version.
One might argue that immutable data are normally stored in Immutable Data objects. The problem is that their names are derived from the content, contrary to the name of a SD which is independent of its content. This name independence is convenient in some kind of apps managing object lists. The fact that these apps cannot prove that the objects they manage have not been tempered with is a regression.
What is the community opinion about this problem? Is it a sufficient concern so that Maidsafe restores previous way of deleting SD? (maybe with some exceptions like tag 7 if safecoins need real deletion for recycling implementation).
Related to this, how do you know if you reached the end of a linked list or if it’s just that the list has been broken? Dealing with linked list that can be chopped off anywhere will be much more difficult.
Imagine our forum is ported to safenet. Each of our forum posts is stored in a separate SD with a certain name (how the name is computed doesn’t matter). The first version of a post is stored with version 0. Each time the author edits the post the SD content is updated and its version is incremented.
I write an innocent technical post about a network functionality and later a forum member accuses me of having insulted him in this post. I cannot prove that I didn’t do that, even if the post is still at version 0 because I could have deleted my post and then recreated it and even if versioned SD are used because I could have modified history in my rewrite.
We could use SD without owner field to prevent modification but in this case the ability to edit a post is lost
Rewriting history of a forum post is not important, but there are domains with more serious consequences (contracts, exchanges, …).
Edit: Used strikethrough to correct claim about empty owner field (see below)
Well, the burden of proof is on the accuser. The way to resolve this is let the forum implementation include functionality to copy the post in question plus it’s signature when you reply to it. That way the accuser’s post includes prove you did in fact say that.
Anyway, if I recall correctly the versioning SD functionality is located in the client launcher, not on the network level. So I think versioning wasn’t something to rely on before either.
But then the chain is at the mercy of the owner of each SD. Now anyone can break a chain if they feel like it. Or maybe im missing something.
Edit: or, require that the sd used as a node in the linked list is immutable but instead of putting the real content inside it you place a link to a mutable SD which you can change as you wish. This way we can have the best of both world.
I am sorry, I did some tests and found that an empty owner field means that anybody can modify or delete the SD. I guess @Seneca has to correct a bunch of replies that stated that an empty owner field makes a SD permanently immutable. What is strange is that Maidsafe never corrected this common belief (in my knowledge).
So I think that Maidsafe should revert to previous delete implementation with a post having an empty payload. That way a SD having version 0 is a proof that the SD was never modified.
This is not perfect yet because we have to trust the app that it only generates such SD objects but this is easily verifiable and it is in the interest of the app to do so because if it is caught generating a SD with a version > 0 then all trust will be lost.
Another way would be to change implementation of empty owner field, but I suppose this is a feature needed in some cases (for example for shared modifiable directory).
I found a simple solution: to prove that a SD has not been modified since its creation and cannot be modified in the future the owner field has to be filed with a constant value with no known associated private key. I propose this constant to be the hash of “David Irvine” string.
And so there is no need to revert to previous delete implementation, current implementation is fine.
We’ve just been discussing this in-house, and we’d like any feedback on our initial thoughts to check we’re not missing any valid use-cases.
I guess the first point is that all of the following applies only to non-reserved SD types (i.e. those with a type tag above 10,000). For reserved types, some or all of this might apply, but there is scope for divergence from the “normal” ruleset if need be.
So right now ownerless SD is supported, but we can’t envisage a need for this, so we’re proposing to disallow SD with no owner. This means that there must be at least one owner when initially putting a new SD to the network, and when posting a new version which changes the owner list.
We also plan to enforce that a Put request for SD requires the version to be 0 (this is not the case just now).
Finally, given the above, the current implementation of delete seems reasonable. That is, the SD is actually deleted from the network and can be re-put by any other user.
Hi @anon86652309, with the current implementation, how is it suggested to create a linked list made of SD? Are immutable SD(is this a thing?) also deletable?
/// Tag representing the Versioned Directory Listing
pub const VERSIONED_DIRECTORY_LISTING_TAG: u64 = ::core::CLIENT_STRUCTURED_DATA_TAG + 100;
/// Tag representing the Versioned Directory Listing
pub const UNVERSIONED_DIRECTORY_LISTING_TAG: u64 = VERSIONED_DIRECTORY_LISTING_TAG + 1;
Respectively 15100 & 15101, that’s strange!
I don’t agree. If delete is implemented like a Post with an empty payload and a version increment (like it was before) we would be sure of the following fact:
a SD at version 0 has never been modified since it’s creation
If it is implemented by an actual deletion from the network then we are sure that a SD has not been modified in the past, only if three conditions hold simultaneously:
version is 0
the owner field is filed with a constant value with no known associated private key (like hash of “David Irvine” string)
we are confident that the app only generates such SDs
The problem is the last condition: it has no place in a decentralized network which should be trustless.
Is SD on the kademlia DHT? If so, DHT data should be considered volatile in my opinion anyways. I mean a peer’s address and what not is “deleted” from the DHT too, right? Sorry if I misunderstand.
Vaults are deleted but immutable data are not, so no clear winner among entities of XorName space
SD can be deleted, we are only debating how deletion is implemented. I prefer previous implementation because current one allows misbehaviour of bad actors.
Yes it is. You only have to set the owner field with a value having no known associated private key. These immutable SDs are also non deletable. This is true in both deletion implementations (current one and previous one)
That’s just plain wrong, a bug and needs fixed. All the reserved types are not currently reserved in code but will be (it’s an esy thing that needs focus).
Yes, but you can still do this.
A delete for instance though could allow a refund to be made of the payments to store it.
Unless you are seeing that delete being delete is wrong in some use case. If you are can you expalin the use case and why delete actually deleting would prevent that?
Using throwaway accounts could do what you say, even for type 0, but it seems you have another sneaky plan here. I am missing it and this is what we are looking for in these API discussion.
@anon86652309 was talking about non-reserved SD types. So I am, and they exclude safecoins and accounts which are managed by core code and may have special behaviors.
Regarding the use case: any application that needs to prove that the SDs it generates can only be modified by incrementing their version. For example a forum in which each post display a number of edits that cannot be forged. A special case is version 0 which indicates that the post has never been modified.
By default that check is made. The process involves at the moment upgrade via only valid successor, amongst owners and previous owners it also checks for valid version increments (by one).
By default yes. But the application can cheat by deleting the SD and recreating it, restarting at version 0. Previous implementation didn’t allow that.
I see, but that would mean the owner deletes? The app itself may not need to create the SD? Could it be something else is missing here, such as the identifier should be what increments? For comments I am not sure, need to think more on much of use of SD (way too deep in other parts right now), but this is exactly why we are here
IF you use identifiers and type tage. Your app selects type # (say 9999). and for a comment use the identifier as the story identifier (or similar). Then commenters upload an SD type 9999 with an identifier, ++1 of your story.
Then users pay to comment, but of course they can delete their own comment. Your app can ensure all comments are version 0 though to stop people editing them.
Early days and crude but perhaps a path to further investigate.
How does the network know if there is a private key associated with the owner field? Does it make a request to the network to look up for a specific public id (not sure this is the right term) and if none is found then the SD is flagged as immutable?
As an aside, I don’t know how we’ll create unbreakable linked list at the app level (or is that necessary?). It seems like an app will always have to guess if a linked list is just cut off or if it reached the end of it.
For example, a forum app requires that a node(a user comment), to be consider valid, is immutable. If someone creates a node that isn’t immutable, the app knows it needs to skip over this address and go to the next one. If there is 10 invalid node in a row, it will skip over them all to the 11th one. But then the owner of the invalid nodes deletes them all. How would an app know if the linked list is just cut off or if it ends? How can an app prevent this? Or does it needs to?
I’m not proposing anything, just trying to see what other dev have in mind to handle this case.
