Uploading to the live network, best practices

I might want to try to upload bigger files and folders, but I am held back by the repeated payments in case of failure. I noticed that Anthill is keeping track of successful uploads and can continue from the failure point:

But:

Is there any upload tool, that would help uploading just the general public or private chunks, and keep track of failures, mitigating the “repeated costs in case of failure” -problem?

Paging @happybeing @Josh @Traktion @Nigel… who else? So many active devs nowadays :grinning_face:

6 Likes

Hey Toivo !

I’m the author of MutAnt, just to let you know that I’m already working on the next version that will try to include public uploads that will benefit from the same resuming system :slight_smile:
It will be still based on scratchpads tho, for the mutability capabilities

However, that system is still quite young and I would advise - if you try it - to exercise caution because I myself still often struggle with the payment system on larger reservations. It will stabilize over time and with more people helping me testing it, and with updates of the network of course

11 Likes

OK, that’s great too :+1:, but I’m really after immutable data now.

3 Likes

If you use dweb to publish a directory it will retry until each step succeeds. So not quite what AntHill does but similar result providing each step goes through eventually.

Also, the REST API could be used to build a web app to do this kind of thing but not on my priority list atm.

I love that @Champii @josh and others are working in this area. :+1:

6 Likes

I am not too concerned about it, I like to make my life harder and only work/test with the live network. I think that I used a local network once this month.

So all my (constant) testing while working on my stuff costs me, it is negligible I if i spent 30 bucks in total so far I would be surprised, most likely closer to $20, I doubt it is even that much.

Just my perspective obvs but at the moment fees are not a concern for me.

7 Likes

I’m thinking about uploading in the range of 10GB -30GB.

If I remember right, the price of 1 GB is about 7$ on average, depending on Arbitrum network prices. So my uploads would cost in the range of 70$ - 210$ if successful. That is already significant money for me. If I’d had to pay that several times, it’s a no go.

1 Like

True, but I suspect/hope that when we can upload data in that range it will be far more reliable and less error prone.

1 Like

Now that the client finds the closest node to a record every time then the quoting part of the upload should now know if the record has been uploaded.

Not tested but this latest version of the client should be doing that.

3 Likes

But I think we still have old nodes that might have dead nodes in their routing tables. Does that mix things up?

But, in principle, once the quoting is reliable, then the repeated retries should not lead to repeated charging? Cause no matter how well the network works, Im 100% sure my router is going to keep on doing it’s thing.

Maybe it’s just a bit too early to try that big uploads.

Maybe it’s better to support native token development with that money and then upload for a fraction of the price :wink:

7 Likes

I did a quote test the day of the upgrade and it got the 7 closest 3 times in a row, except it found an 8th once

At that time not many nodes would have upgraded by then

Just test a few smaller files and see what happens. The amount used should reduce each attempt

4 Likes

Nah…

My plan is to upload something that can be advertised being available way wider than just in this forum. The download performance seems to be quite something already, so I hope that would be a way to get more people interested.

“Here is this 10Gb piece of valuable information you all want. You just need to download it from the Autonomi network. Be careful, the dowload is so fast, it will blow your router through the wall into your neighbours property. P.S. the download link works forever.”

Then people flock in, ANT goes to 10$, and we can fund the native. Well in my dreams at least. :grinning_face_with_smiling_eyes:

13 Likes

I’m sorry I don’t have anything to contribute to OPs original question, I just wanted to call out that name as it made me groan at the pun. I love it.

8 Likes

If you’re going to spend all of this time and effort on an upload, then it should be the most mind-blowing upload possible. I’m sure the community here would enjoy giving you feedback on what this amazing upload should contain.

I think you should upload a book titled, “The accumulated unique wisdom of Toivo, Volume 1.”. It might not be 10 GB though.

2 Likes

It cannot be put into a book. It needs to be a video. Only couple of things there, but so deep wisdom, that it needs to be extremely slo-mo. Hence the size.

2 Likes

“The Large Red Upload of Toivo Thought”

Download it NOW or your loyalty and commitment to Autonomi and the Glorious Cause of Internet Freedom will be doubted.

And we don’t want that, do we? :rofl:

3 Likes

“List of things everyone agrees on.” Again not 10Gb. Probably not even a 1b.

2 Likes

Oh I think deep down, its a LOT more than that :slight_smile:

We just disagree on how to get there :laughing:

1 Like