The purpose of this topic is to track, day by day, the experience of an end user (me) who uses Dave to upload data to Autonomi.
Day 1
Check out the Impossible Futures!
Day 2
Check out the Impossible Futures!
Day 3
Check out the Impossible Futures!
Day 4
Check out the Impossible Futures!
Would be cool if you revisited some of the uploads now and then for repeated downloads to find out if the data is truly permanent.
I’ve uploaded loads of stuff for IMIM through CLI or REST and it is all still there. Has been since launch.
Just FYI, we also have a list of files and directories we download continually to monitor the performance of downloads on the production network, but it also serves as an indicator of data integrity. One of the downloaders uses the --quorum all option, which downloads all replicated copies of the chunks.
So far, no data loss has been indicated. The network has actually been pretty resilient considering the amount of churn and fluctuations we’ve had in estimated node size.
its pretty amazing how well she has performed after all that has been thrown at her.
if anyone wants to try downloads here is 50Gb of files to play with
ant file download --retries 20 3b7f320f24e9f2274ea6917f9dc34f5df3386123c70b3fd3b2161fcdf6c0505f ubuntu-16.04.6-desktop-i386.iso #md5sum feefb18e7916c9a16bb09923ed98df64 1.6Gb
ant file download --retries 20 f2e49a420cc6796361fdf836cc573dedf63c3b3b11f93bbd1a15962a9830cd8f ubuntu-24.04.2-desktop-amd64.iso #md5sum 094aefdb1dbbdad8aa99600f8413789b 5.8Gb
ant file download --retries 20 ccc401961cc29a0d9fce5e6dd10ff4107d594485678e17fedcdd545ae2f1934d ubuntu-25.04-desktop-amd64.iso #md5sum d79522031c673fe5ff1c037038806012 5.8Gb
ant file download --retries 20 aaeb6271144a7bb4529a08e580062d9de7b7a1bb7ec6ec9b5498deb7afb94035 2022-07-01-raspios-bullseye-i386.iso #md5sum e57f42852306ac5ed0d9e97bdc3513cb 3.4Gb
ant file download --retries 20 a9bdf8ce7423d6a14b373db5d056beec8dd91227d3e5184f824e8359fdbadad8 2025-05-13-raspios-bookworm-arm64-full.img.xz #md5sum 7bf17963c4fc2d8716f53f89c90c0fba 3.1Gb
ant file download --retries 20 e990f4c13b3978f26f0d6531086f24a92c8dfc22283a36888b450d3c78457623 Fedora-KDE-Desktop-Live-42-1.1.x86_64.iso #md5sum 1db4bd673e0cc72202578b6c2296e485 2.7Gb
ant file download --retries 20 e42dca760e71158aacea4b6f71e33de861fb02901c16c13ecfd0714337cad454 Fedora-KDE-Desktop-Live-42-1.1.aarch64.iso #md5sum 531804fcc615d34cb130a2623e64fdf3 2.6Gb
ant file download --retries 20 33cc094b8386605c9c61f73c4629f21984bf070625684366c8c926fda34cfd19 Fedora-KDE-Desktop-Live-42-1.1.ppc64le.iso #md5sum 3b9c846cf4369bc15f2902afa3eae020 2.6Gb
ant file download --retries 20 e322cc021ecc9f5c51d7ac73786ffcf0244d52ff41c48e1a12594833ffb33995 livegui-amd64-20250720T165240Z.iso #md5sum 338eadd1a40dc4a71963547e34905f83 3.7Gb
ant file download --retries 20 9f1c51fd6eb12be85dadc4815c5b3cec07e899826fbeb907d34369d56384c746 2025-05-13-raspios-bookworm-armhf.img.xz #md5sum f1cdb4cb7e1d0d1fa24ea2e0468537e5 1.2Gb
ant file download --retries 20 b1a16a0d7b14e0bfc91511dcdcb6b4fefcf428aa768ae37d40dbf29c51efc5c0 2025-05-13-raspios-bookworm-armhf-full.img.xz #md5sum f43d8bc191501491148f302cdfc73d29 2.8Gb
ant file download --retries 20 58f546c93e031987d5a13e12d23da283be479debb9cba148e39bb8d2c68a983a linuxmint-22.1-xfce-64bit.iso #md5sum 7ed0eb51afeb3009aa042bbbfc902474 2.7GB
ant file download --retries 20 0b4f7d21d932c573c99cee9811f951c320aa38277660ad4dd70855d870de534c debian-12.11.0-amd64-DVD-1.iso #md5sum 39a31c7218b4606cf7f4408c0b261a2f 3.7Gb
ant file download --retries 20 81c97fd2385d3dc0c9c17a61ec393386480fd46c86989b7cbbf82839c467a6c2 debian-12.11.0-arm64-DVD-1.iso #md5sum 916173947d0507c673a36b6950482f05 3.8Gb
ant file download --retries 20 e751cb98326ba5e11b5e0cf3dc02e96e3b53d30819247b1f230072907c21d9b7 cachyos-desktop-linux-250713.iso #md5sum 7c84485fc91ec645b6235727ff4adcef 2.8Gb
ant file download --retries 20 4ea3a7d2300aa96b03009e84443a0923fabb688fe0ef14c48c87e5281319e82a cachyos-handheld-linux-250713.iso #md5sum d1354a944600d9088540c18da07add78 2.6Gb
Quick query, is that just from the 5 closest nodes or the upto 50 (i’ve seen in tests) nodes that have the chunk still in storage
I don’t know.
Hey @qi_ma could you pick this one up please?
to be accurate, --quorum all means at least 5 copies fetched to qualify the download as succeeded.
Thanks for that. So not a way to find all the nodes that have it. After a few more updates i might redo my tests to get an idea of how many copies the ends up for new records
This is a DHT de-centralized network, in theory, no way to figure out ALL nodes that holds the copy.
Unless you ask ALL nodes one-by-one
Ah yes of course. I just wondered if all nodes with the record would respond to the request.
Although for a new specifically known record I am able to determine the nodes storing that chunk. Well nigh on all as there might be some quirk or random occurrence that stores the record elsewhere.
the record for sure could be stored on some other nodes elsewhere.
If a node stays with same ID on the network for long time, the records on it could have many that no longer being close to it anymore, i.e. with more nodes joined network along time.
hence a record stored on non-close node somewhere
such records will only got cleaned-up by the host node, when reached some threshold, say accumulated to certain level.
Day 5
Check out the Impossible Futures!
Big difference in ETH price there. Are you going to try to get them up cheaply, or just uploading with the price as it happens to be?
Up to $1 I consider it a reasonable price, anything above that will make me postpone it for the next day.
Check out the Impossible Futures!
This is good data to have. Much appreciated. Would you also post the miniscule amount of AUTONOMI paid so we can get a percentage of gas vs AUTONOMI cost?