Some notes about the recent changes, some not quite public, in both the Autonomi APIs and dweb, which should make uploads and downloads start to work again.
Upcoming fixes have improved downloads
Both @Traktion and I have tried downloading with the improved Autonomi client API which are in the latest release candidate but not yet the ‘stable’ release (so not yet in ant unless you build yourself).
I used dweb to download and inspect the content of a directory both with and without these tweaks.
I was downloading a small directory (which @Southside created for his website) and found that instead of 3 of 5 downloads failing, 5 out of 5 succeeded, although they took a loooong time (1 to 3 minutes). @Traktion reported something similar.
This improvement in reliability is very good to see. I still can’t do anything over mobile b/b hotspot, but if you have a good connection and on VPS things are improving.
Striving for uploads to publish websites
I haven’t had a chance to try uploads, but have also added code to make these more reliable in dweb. One is that when an Autonomi API call fails dweb will immediately try again and repeat this until it succeeds, or the universe ends. You can though set a limited number of tries if you don’t want to wait that long.
There is also a problem when trying to upload a directory. When this fails the Autonomi API (for putting a directory) starts again from scratch every time. This, and the fact you get charged for repeat uploads means that even a small error rate can mean it is very unlikely to succeed, plus, your wallet is being emptied much more than necessary every time you retry. Individual charges are small but will add up when dweb keeps repeating this over and over, so…
Upload file-by-file
I’ve also added the option (currently the default in dweb publish-new and dweb publish-update commands) for dweb to upload each file individually.
This, combined with the retry feature means that even if only a small percentage of PUT operations succeed, it will be possible to publish a directory or website (using dweb) if you wait long enough.
I’ll be trying this once the release candidate makes it into a release.
Inspecting a History and its Pointer
BTW dweb also has inspect commands, one of which shows you whether a pointer is pointing to the head of a graph for a History or is stuck pointing to an earlier entry.
This is another issue we have when using the History to implement versioned websites. But, again dweb now has a feature to work around this by ignoring the pointer if you want and scanning the graph from the Pointer target to the end (real head entry).
Uploads, downloads and web viewing may be close
All-in-all, it should be possible to publish directories and websites with dweb so long as a percentage of API calls are succeeding. This approach will mean upload and download will be very slow for now, but that’s better than not working well enough to publish and view websites at all.
This may be improved even more when nodes catch up with the last two releases.
Fingers crossed for next week that we’ll all be able to try out some of these features.
EDIT: all the above features of dweb are now available in dweb 0.3.1 so:
cargo install dweb-cli
and by all means beat me to seeing if it will upload, but watch your wallet if you leave this running! It will keep going and going until all the Autonomi APIs have returned success!