I really appreciate you joining this discussion. Thanks for correcting me on the in memory feature - not sure how I missed that as I created the issue requesting it. I shall go dig into that.
My questions are really about having a standard API call which avoids unnecessary chunks for small files, to provide a standard way for all apps to access files (store + download), consistent content hash (datamap address) which could reduce 3 chunks to a single chunk for many files and reduce storage costs and potentially the load on the network considerably (say 60%). Leaving this to apps or for later doesn’t achieve those benefits as far as I can see.
I understand this is not a priority but am not sure why, and don’t see how it can be implemented later - as a core API - because that would break content addressing etc. Which means it would be app specific and lose many of the benefits.
Am I correct here or is there something wrong with my reasoning?
I realise I may not understand all the implications, but given the potential benefits I would like to understand either why this isn’t feasible or how it could be done later without losing most of the benefit.
As you say, we currently don’t. Although we have lower level API that would allow one to upload the data directly in a chunk, bypassing the datamap and the 3 chunks minimum. But it is not the standard.
The 3 chunk minimum is required for self encryption, any smaller than that and we would be forced into un-encrypted data.
I think your claims are legitimate, and especially so if we’re talking about public data.
As you guessed we have plenty of other priorities atm, but will try to keep that in mind.
Thanks for correcting me on the in memory feature - not sure how I missed that as I created the issue requesting it. I shall go dig into that.