Possible to download only file metadata?

Development
1

I was reading through the API docs and it lists the public archive struct like this:

pub struct PublicArchive {
    ///           Path of the file in the directory
    ///           |         Data address of the content of the file (points to a DataMap)
    ///           |         |         Metadata of the file
    ///           |         |         |
    ///           V         V         V
    map: BTreeMap<PathBuf, (DataAddress, Metadata)>,
}

and the Metadata field like this:

pub struct Metadata {
    /// File creation time on local file system. See [`std::fs::Metadata::created`] for details per OS.
    pub created: u64,
    /// Last file modification time taken from local file system. See [`std::fs::Metadata::modified`] for details per OS.
    pub modified: u64,
    /// File size in bytes
    pub size: u64,
    /// Optional extra metadata with undefined structure, e.g. JSON.
    pub extra: Option<String>,
}

I didn’t see anything in the API docs that would allow someone with the address to a public archive to download the metadata only. Is there a way to do this?

1 Like
2

I don’t think there is a separate datablock with metadata. Isn’t all part of the one datablock being stored (maybe chunked, prob is)

3

I could see it being useful in the same way as HTTP’s HEAD is.

4

That makes sense. I wouldn’t expect a separate chunk for metadata, especially when this is optional. It would be handy if this was at the beginning of the first chunk stored, then you just need to download the first one to read this stuff out.

5

still need 3 chunks to decrypt one chunk

6

oh, good to know. I didn’t realize that.

Edit: Is that for archive_get call you need 3 chunks or does that include the native chunk API chunk_get? Where is this documented?

7

For encrypted data via self encryption you need 3 chunks for self encryption, the default, and you need 3 to decrypt. Self encryption cause it to be 3 chunks minimum and that is documented with self encryption

8

OK, I see it now in the datamap description:

DataMap
Holds the information required to recover the content of the encrypted file, stored as a vector of ChunkInfo (list of file’s chunk hashes). Only files larger than 3 bytes (3 * MIN_CHUNK_SIZE) can be self-encrypted.

I didn’t pick up on the importance of the 3 in the description.

1 Like
9

Self encryption process involves first chunking up the file into at least 3 chunks with a max size of 4MB (currently) and then using the chunk before and after to generate a encryption key to encrypt the chunk. First chunk uses last, itself, and 2nd chunk to derive the key. Last one uses the first chunk as the next. And all the others use the previous, itself, and next chunks to derive the key

Thus the 3 is important for encryption, minimum chunks and decrypting

1 Like