Can MaidSafe provide a summary of storage features for the MVP (i.e. access control, revocability of access, deleting private data etc)?
So far we have public immutable data and shared mutable registers, though I’m no longer clear on the precise features of the latter, or on what will be possible in the area of private data which is now I believe the main focus.
As I understand the MVP, we won’t get the Safe App, but that this is still planned to follow. So will the MVP aim still to provide the same underlying features for private data that have been described for the Safe Vault App, or if not then what subset?
This would be useful in helping understand what kind of app can be built on the MVP.
It’s an exciting time to start thinking again about building on the Safe Network.
We are close to being able to do that. Here are some thoughts
chunks are immutable
registers are immutable
All data is currently private (but you can share out of band links to files/directories)
No data is deletable
Something we may also require
An anything chunk → i.e. an account packet that is NOT concurrency safe, but deleteable.
This would allow folk to have a bucket they can put keys/directory structures (root structures) and more in. It would not be CRDT, so it would be fully editable by the user.
I am ignoring the currency here as that is a system in itself, needless to say we will release the API and tools (like the recent audit tool) for that. What I would like to see is us producing a hacker kit that shows folks where we think currency attacks could happen and let everyone try it.
Immutable? If so can you clarify as these have always been called mutable haven’t they.
I’m curious about this as it has been much debated in the past and I assume would make it impossible to revoke access which would be a useful feature when sharing private data. Perhaps the reason for…
Is this effectively a third data type? Can you say more as with the above it seems very significant.
N.B. I agree the token is a separate thing, not for this topic.
What I mean is you can consider them append only. So they are mutable as they can grow, but they can never shrink, if that makes sense. So the entries are immutable, but we can add more entries. There must be a better name for this?
The problem with delete is
For concurrent data, we need to keep delete sets or seperate copies where the delete set is queried, then there are issues if you want to undelete a delete etc. So it get’s messy unless you create a huge data type where deletes have ancestors 9logical clock) and so on.
Yes
This would be a private type only for you. A login packet or scratchpad, but with no concurrency protection. So if you overwrite it from one device then it will be overwritten and if another device writes back the original then, you are back to the original.
It should not be shared, but folk may well do that as we cannot stop them. I see it restricted to one suck packet per user though as it could be badly misused otherwise with errors galore.
In summary then we have (please correct/clarify if needed):
forever storage of immutable data
forever storage of concurrently appendable versioned registers (based on CRDTs)
a re-writable structure per ‘account’ that should not be shared because it is meant to hold sensitive information about the overall data
EDIT:
immutable data files are chunked and encrypted if they are larger than a threshold size, but small files are stored without encryption as a single chunk/record and so are readable by those with access to the nodes storing them.
This supports:
a single private read/write file which is held securely on the network (non-concurrent write)
storage of immutable files and directories that are private by default, but can be shared by sharing something???
Limitations:
once a file or directory is shared, access cannot be revoked and the data remains forever accessible to anyone with that access
The last point troubles me, especially with the focus having moved from public datasets to private data. Is there a way to allow sharing that could be revoked which can be build on this? I can’t see it, although I’m not sure how it would have worked before either!
It also feels like we may have lost some of the functionality needed for Jim’s Safe Vault, and the macaroon’s style ‘capability’ features that Joshuef explored. Or can they still sit on what you’ve described?
I guess you can build anything if you build an encryption layer on top, but then maybe you’d lose some of the valuable capabilities on the core data types. Just speculating!
I’d like to poke at this from the point of view of what kind of features different application logic would need and so what kinds of things can be built and how they would be implemented. For now it feels quite restricted.
Maybe those who have been contemplating building apps can comment on features they require in the API? @seneca@nigel and I’m sure there are others but I don’t recall them.
I have always had this thought. If you share for even a millisecond, it’s shared and cannot be undone. Revoking is saying you can no longer access this stuff, but you already have. So the best (honest) approach is a new register shared with a new group.
Great shoutout, we need as many voices as possible
That’s something you would need to consult with our very own @loziniak!
I was waiting for a more formal post but now is as good a time as any to announce that @loziniak has been hard at work plugging away to prep the JAMS MVP for SN API’s recently!
We’ll see how things come along but hopefully more to report in the not too distant future. Give it up for the great and powerful Loz
I left some comments on the dev forum. The API certainly needs polishing from my point of view. I have some thoughts about it, but I kept it for myself, because I have an impression, that it’s not a priority for the Team at the moment and I hope it gets more attention soon.
I would love to see a possibility to check previous versions of register data. We had this in earlier API.
The conversation here reveals some interesting facts, mostly about data security. Seems to me like a similar case to everything-crypto well-known fact, that your data is as secure as your keys. Here we can say your data is as secure as the links to it.
From the other point of view, even in law we have the right to be forgotten in EU, this seems to be really important for many people.
What is also worth mentioning, Safenet’s registers can be restricted if it comes to writing. Each register has a list of keys that can be allowed to write, or a permission to let anyone write. I wonder, if this new “unsafe” data type could also be restricted for write?
This may no be a problem in a level of network itself. But it seems like a big problem for companies who could utilize Safe Network as their backend, if it’s impossible to fulfil their legal obligations with this network…
Or not. I’d be surprised if Google for example deletes data, and expect it just excludes certain results from search. I’m not sure if that’s legally compliant though, so may be wrong.
So it becomes a question of what defines deletion of data example writing over it with random letters or removing the marker to the data like standard deletion on a PC when you delete something from the trash bin.
Quite a grey area I think. Mostly, according to this article, it seems that de-indexing is sufficient in the EU at least.
In relation to such processing activities, the data subject has the right to the de-indexing which is understood as the right to have the information referred to him and published online “ not linked to his name from a list of results appearing after a search made from his name”. This is a fundamental right, which therefore takes precedence over the economic interest of a search engine operator and also, over the opposing right of publicity of news.
@josh has reminded me that we’ve discussed this before and it still troubles me. I think the outcome of those discussions was that it is up to applications to ensure encryption of data, but this seems unsatisfactory, especially if the MVP is to be marketed with the emphasis on private rather than public data.
We appear to be planning to launch a privacy focussed network without a way for users to store their data in a way only they can access. I can’t go along with that, but maybe this is not the plan, IDK. @dirvine?
I would agree with you on not going for that approach
The typical and default use case is users use the client apps that use the API in it’s default settings. That means all data will be passed through self encryption with registers pointers to that data etc. Unless people specifically and purposefully give others access, then their data is fully private and encrypted.
For folk to put up data that is not encrypted should mean they don’t use the API / self_encryption etc. but we cannot stop that.
It’s purely up to users if they wish to share registers and make them open, by default they are private and the core network does not provide a way to make them public or shared, although it’s easy to do that if users wish it for their own data or subsets of their own data.
tl;dr Default is all data is private and fully encrypted.
If folk want to share data they can share links and client apps can help them do that too, we can also not stop that.
EDIT - on the very small files not being encrypted. I think there is confusion there. It’s true they are not in themselves, but they are part of a directory and that is encrypted, so they are actually encrypted in the directory structure. I hope this makes sense, I will try a wee diagram
big file → chunks out + data_map
wee file _. Contained in data_map
Dir → file1_data_map file2_data_map file3_data_map
->SubDir → file_4_data_map etc.
The above line is all OPEN DATA, i.e. unencrypted data maps. However, the dir size is not small, it’s well over the self encryption levels (just a couple of data maps would do it, probably even just 1 data_map). So this structure all goes through self_encryption and you are left with a single data_map. That is your root directory, and unwrapping that gives you access to the whole filesystem that is only yours. Tiny, small and large files all held encrypted.
It does seem to be a pretty obvious Achilles Heel in the self encryption process. Small chunks, or small files, which are likely to be text files and contain readable info, are not encrypted and uploaded as is. This has troubled me for a while now. It shouldn’t be left to the app to sort out.
