I had an idea… I know
I suggest it will be best to keep discussion on github but posting here for visibility:
opened 10:40AM - 11 Apr 25 UTC
FYI: @traktion @loziniak
### Background
I've been modifying dweb to display … websites and directories from not just a `PublicArchive` but either that or a `PrivateArchive`. To achieve this I made a wrapper that holds one of each, can deserialise either just from an address, and provides a simple but unified API.
While doing this I saw that there is very little difference between these types and having anything in posession of either struct, has full access to the files (whether the struct is a PublicArchive or PrivateArchive).
So it seems unnecessary, wasteful and a bit misleading to have separate structs. In fact for many use cases (e.g. serving websites, sharing directories) it will be best to use a PrivateArchive - even for public data - because of efficiencies in storage (and therefore cost) and speed - because you save one chunk per file by storing the datamap in the archive.
### Proposal for discussion
Create a single Archive type that has two maps, one for private files and one for public. These correspond to the slightly different map in each of PrivateArchive and PublicArchive. Then provide APIs for accessing each map separately and some additional features on top of that such as `make_file_public()`.
This also simplifies sharing, so an app can show the private files and a use select which to share. As their datamaps are put on the network then can be moved from the private map to the public map.
Also, I think all files, public and private should have, or to least be able to include their datamaps in the Archive. One way to do this is to just store the datamap, but have an API for getting the content address for any file in the public files map.
The client APIs related to archives could remain unchanged apart from returning the new unified `Archive` type which keeps the semantics/understanding of what is happening clear and minimises work updating old code for the change. So `dir_content_upload()`, `dir_upload()` and `dir_upload_public()` etc all work with the same underlying `Archive` type as now. All changes are hidden under the hood until someone wants to access the public or private files in the archive.
Have tagged some on github I found easily and for those I didn’t…
@riddim didn’t find your handle there easily (at least not you I think)@safemedia
6 Likes
Kindof related for anyone creating directories online using archives, so also for publishing websites… (FYI @Traktion , @riddim )…
I’ve just pushed (not yet published) changes to dweb so that it:
stores websites using PrivateArchive instead of PublicArchive (this doesn’t make the sites private, but saves one chunk per file and speeds up access because you have one chunk less to fetch per file - the datamap chunk).
can access files given the address of either PublicArchive or PrivateArchive (@Traktion it would be great if you could also support this even if you don’t want to publish data using PrivateArchive. Then both our apps will read data published using each other. Reading either is a little tricky you you can find the code which differentiates between one and the other here ).
5 Likes
Forgive my ignorance, but do you not need the datamap chunk to list the files in the archive?
I’ve not played much with PrivateArchive, but I assumed you wouldn’t be able to list the files in the archive without the datamap?
1 Like
I’m referring to the datamap for each file in the archive.
So yes, you still have the datamap, or rather the data address for the archive. That can point either a public private archive.
1 Like
