An Overview of the New Data Types

[dirvine]

@neo

What if I “publish” my data later. What happens to the ID or is it uploaded again and I pay twice?

Yes that is currently what would happen.

[neo]

@dirvine

Yes that is currently what would happen.

So does the ID encrypt the data? or is the ID stored as part of the data and thus hidden?

[riddim]

If it was part of the data and the data could be encrypted the sections/vaults would need to be able to decrypt random data and inspect the content Oo

Edit/ps: burdon+danger… So more an anti feature… It must be part of the data type itself I would blindly guess and (private data types) possibly have their own address space apart from the ‘regular’ xor names …?

[dirvine]

@neo

So does the ID encrypt the data? or is the ID stored as part of the data and thus hidden?

It is in the RFC I think, anyhow the name == hash(content) where content = (chunk + id). So the ID is prepended to the data and the name is the hash of all of that. This makes it unique to the owner

[neo]

@dirvine

where content = (chunk + id). So the ID is prepended to the data and the name is the hash of all of that

So in theory you could still (sort of) publish the file by sharing the data map

[dirvine]

The network will not allow anyone to download it, only the owner. So you would need to give everyone the private key to do that.

[riddim]

Do I understand this correctly? The private key needs to be presented for downloading - so you are asked to provide proof it’s you by signing something and to the content the public key is part of the data +the network reads that data +checks your signature to verify it’s you…?

[dirvine]

@riddim

Do I understand this correctly? The private key needs to be presented for downloading

No

@riddim

so you are asked to provide proof it’s you by signing something

Yes the Get is an OwnedGet where you need to sign that with the private key. The vaults check the sig against the sig in the data as well. So you will only get this data if you can sign the request for it, if that makes sense?

[riddim]

Nice and simple mechanism that uses the existing infrastructure

edit: (and when owned by an alias can be shared in an easy manner with a certain set of people) - I like it

[dirvine]

@riddim

edit: (and when owned by an alias can be shared in an easy manner with a certain set of people) - I like it

Yes I think we have not included this in the RFC or made it clear, but it is a valid mechanism for sharing amongst selected members. With BLS we can even enforce n of p sig schemes and so on, but that is for later for sure. The set of users with the secret key though is no code and can be put to good use straight away.

[riddim]

Maybe makes sense to clarify that people are not necessarily people but might be applications as well - because I probably want to share access from my photo app to the photo app of my mum but not to her Sudoku app

[Traktion]

@dirvine

Yes I think we have not included this in the RFC or made it clear, but it is a valid mechanism for sharing amongst selected members. With BLS we can even enforce n of p sig schemes and so on, but that is for later for sure. The set of users with the secret key though is no code and can be put to good use straight away.

Great to read this, as I haven’t heard private groups discussed in a while.

[Traktion]

@neo

What if I create ADs for indexes to data. The index ADs need updating and it would be good to delete old useless indexes to save storage space.

But since the indexes have to be accessible publicly then they can only be ever appended to and thus grow. If the indexes are changed regularly then this represents a lot of storage that will never be accessed again as it is only indexes and no longer pointing to data.

If we have a perpetual web and data, should these indexes ever be deleted? Surely, old indexes play a role in maintaining fast access to historical versions?

[neo]

@Traktion

role in maintaining fast access to historical versions?

Many of these would use a different way for backup that allows different ways to access it.

These database maybe private to the company and the company decides how to hold the data

But yes for a range of databases mostly used publicly would want it to maintain the ADs

[happybeing]

@neo

What if I create ADs for indexes to data. The index ADs need updating and it would be good to delete old useless indexes to save storage space

When to use each datatype is a good question but will depend on the details.

For example, the obvious answer to the above would be to use immutable data which is deletable until published - so OK for internal databases for example (most database use by far I imagine).

That’s fine if delete is free or even gets a refund, but what if it costs to delete?

To make decisions on this we need to try and model what’s best for the network in different scenarios. This will be tricky because scenarios will change according to the parameters we set, which in turn changes what’s best for the network!

I think this is an interesting and complex debate to have!

When is delete good?

I think we will need to have a good grounding for a belief that deletion is always a good, rather than letting junk accrue forever.

I think most of us feel that letting junk accrue forever is bad, but I think we need to show that is actually the case using hard facts, reasoning and imagination first.

Then we can make decisions on whether delete is just good for privacy, or if it is generally good to delete junk where feasible.

That in turn feeds back into the charge or refund question.

[neo]

I see database indexes as an example of an in between form of private & public in that the index is kept by the application for the purpose of the application and in some cases private to the application, but in other public. The application though does not want to store its private key in the application so it can have private indexes. Another application is order entry where the company wants data generated by the application but not modifiable by the user. Otherwise fun and games with the ordering system for the company as users fiddle with stock levels that the ordering program needs to update when an order is placed…

[Mindphreaker]

Ah yeah thanks. Overlooked it when I first read it. It’s pretty clear in the RFC:

If the client wants to publish an UnpublishedAppendOnlyData , they need to reupload it as a new PublishedAppendOnlyData object on the network.

[Nigel]

@maidsafe are these new data types currently getting any docs for use in the API or is it far too early given these are still being integrated into SCL and vaults?

[ustulation]

See if starting with the RFCs help like this one. Maybe not 1-to-1 mapping but will give you a pretty close ballpark wrt APIs.

Similarly there’s one for Immut-data in that repo just below this ^^^ one.

[Nigel]

Thanks Spandan, appreciate the response