[Offline] 50GB Static Network - Round 3

performance is blistering from my end uploaded 40Gb of tunes no problem i forgot to time it but guessing 4 or 5 hours.

only errors I saw in the nohup was when a file was larger than 10Mb

now downloading them to another client at the moment

on a free tier oracle arm based VPS

5 Likes

yep.

total files uploaded:253 – total bytes uploaded:1,110,990,333

I’m on a slower connection than most I expect. but wasn’t having many of these errs when I started, but becoming more common.

5 Likes

The ~17GB images folder upload went perfectly fine. :fire:

FilesContainer created at: "safe://hyryyryzqu8so48ifpqk9nuyw6ymrdc51dxuxcx6nxkd7pek1ht5yfxqkgonra?v=hxqfx7iij9aqakfqbyg6h3xc51mqe5rsbe888g6r77z685hk8n17y"
+---+---------------------------------------------------------------------------+--------------------------------------------------------------------+
| + | ./sn_photos_17go                                                          |                                                                    |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|
| + | ./sn_photos_17go/.DS_Store                                                | safe://hyryyyyenfb13sfibwwwkhko96d13hucitz7sjcsz4xbo1cahw4cas7n1yr |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|
| + | ./sn_photos_17go/018b51ef.jpg                                             | safe://hygoygym9rpyxjpp5mnghuxsmwo6z1pyzo6taitdhm7usetra34uyycap5c |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|
| + | ./sn_photos_17go/025f1dd8.jpg                                             | safe://hygoygyjgn1hx9wobgubqganqn4mb19o619g7y9tp1k4c6w3tk4godzkeqo |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|
| + | ./sn_photos_17go/06.jpg                                                   | safe://hygoygycj9s1aaij3j9zof69gu8jnx3kbafk88z6src5cwb6xorg63rip4r |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|
| + | ./sn_photos_17go/07bd1804.jpg                                             | safe://hygoygyqkpket3cczk7qk8ahrdf3yzafzgeu155wqqzkpnb6pajgghkzrih |
|---+---------------------------------------------------------------------------+--------------------------------------------------------------------|

For anyone struggling to gather a lot of real files under 10mb, here is a text file that includes a ton of image URLs (from unsplash.com) for photos that I didn’t already upload on this testnet :
safe cat safe://hy8oycym7rdmj5h1gy5x9i5hu83of5dqebcp3o7ayky6ercuaqotx7dxkga > unsplash_images_URLs.txt

On linux or mac :

  • Create a folder.
  • Place the unsplash_images_URLs.txt in the folder.
  • open a terminal and sudo apt-get install wget if you’re on linux or mac
  • then cd to the folder.
  • Just use wget -i unsplash_images_URLs.txt on linux or mac, or try for /F "eol=;" %f in (unsplash_images_URLs.txt) do curl -O %f on windows (I didn’t try, I’m on mac)

And it will try to download everything until you stop it (or your hard drive is full like mine…)
Many photos are > 10mb, so when you’re finished make sure to delete these before uploading your folder to this testnet.

11 Likes

I’m going to get some rest. If anyone want to use my gist for uploading random data files under 10Mb - I’ve updated the code to include a files counter and total bytes uploaded sum. The verify doesn’t work yet - leave commented out.
my totals for the evening:
total files uploaded:279 – total bytes uploaded:1241542269

good night all! :wink:

10 Likes

1.7gb (200 random.txt files) uploaded and faster than the last testnet. Going for 6.8gb (800 random.txt files) now.

5 Likes

Iv run out of actual files so now I’m giving your script a whirl nice work :slight_smile:

3 Likes

Personally I didn’t get any error, I was echoing @TylerAbeoJordan and @Southside posts about clienterror timeout messages.

In any case, great testnet! :+1::muscle:

3 Likes

Uped the auntie

Come get me gramer Nazi’s

Now got 3 clients uploading using @TylerAbeoJordan script two cloud based and one home based.

Looks like Internet connection is a key factor as the home client is an i5 and it is being far outstripped in upload speed by the arm based cloud clients.

4 Likes

Ive been very busy all day, just poppin back inside to the kybd from time to time - FWIW I have not seen any further timeout errors since the morning.

I even did for i in {0..1000}; do safe wallet create; done

which completed successfully without errors. On a local baby-fleming, I usually see 2-3 errors per run.

3 Likes

Oh FFS… As the resident Keeper of the Standards, I am duty bound to not only bite but chomp on this disaster

Upped the ante ← sod all to do with yer extended family
Come AND get me, grammar Nazis < what uncouth fellow stole my inappropriate apostrophe?

5 Likes

Good to see the old Island traditions of making your own entertainment during the long winter nights are not entirely lost.

Incest, spòrs don teaghlach gu lèir

1 Like

lol I read that and wasn’t understanding, quietly thinking “what auntie?”
Now I get it… haha (fabulous spelling :smile: love those)

6 Likes

I’m impressed so far. Have been uploading 8mb junk files & downloading them again, checking with a sha256 sum (cos that’s the sort of person I am - sorry :smile: )

About 1min20 for the upload & 4-5 seconds for the download which is much faster than I was expecting (admittedly it’s been a while since I’ve participated in a testnet). No errors so far.

This is really encouraging - a great start to the new year. Thanks guys.

13 Likes

1000 wallets created in 35 mins - no errors

willie@gagarin:~/.safe/node$ time for i in {0..1000}; do safe wallet create;  done
Wallet created at: "safe://hyryynytiis45ujf3qcmtg139go8t4q8k6mztcq81xgrkk1jcjrcpnkfunab6o"
Wallet created at: "safe://hyryynyi854hmckud1zpz77firi9fek1qaz9if13inoeqejgn9n4esz154rb6o"
Wallet created at: "safe://hyryynywbkobcz65ngbe54gxfjqqu8j3igsjhymn5yi4a9wgmmh89xsawwhb6o"
Wallet created at: "safe://hyryynywht9zo9nbbzusfwjk4yes8daq9hcz8yxg1xtzsi7eyfmpgqmfo9eb6o"
Wallet created at: "safe://hyryyny1u6onqifetmr3mz18ctbd9jgo6gycmxcbac4c4ckrdudi5mhboxyb6o"
Wallet created at: "safe://hyryynyw6gx4signa4kc4gz8ojdxs19w1eb9qdg63xxq6abzzb4w4dcacdob6o"
Wallet created at: "safe://hyryynyibo1szckrg1xd4tgua195bi3ypnttf8tnba8qdqmmdg4apw13ibcb6o"


Wallet created at: "safe://hyryynyu3eakw3x1ht7mmi69ra7z8hxghum3rwdrztx4hx55feebp8d9zscb6o"
Wallet created at: "safe://hyryynyijyqknqqybm4ndak3dsfhjegz8hqi3rf8rgjgeznxk4uqxi5o5neb6o"
Wallet created at: "safe://hyryynyskt7t6imxmijqj6n4ktw4tzxofn9pfe6cusnzxm8rc6uzmmj8h8ab6o"
Wallet created at: "safe://hyryynyiwzdppsqkq8nsr5dynyrm7uk19e1hwdjiejdrfh6q7up3789w87yb6o"

real	36m2.938s
user	0m35.528s
sys	0m6.075s

buuuuut…

then I started getting timeouts

+---+--------------------+--------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryuqwbe3qteqr79w6zc9ngfikanox8mzty6c7hzm7ccyxuzrpus7acnra?v=hmnmjte8b5ysotmsb1j3a5s3ddqajhst9p7udrwzx4m49bdi3zbco"
+---+--------------------+--------------------------------------------------------------------+
| + | neal.tar.gz.partau | safe://hyryyyyqj99j753w665iwydse9th6enzyqxiytqqorsgtim8eg4jwh7cnho |
+---+--------------------+--------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryzbhd667cp5igsh6i7jhshnjrgkonue6xwom4iheo4dy8adf9ti9hnra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partav | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 462026..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryt6dk45t41dwhtetsmkjqh6i5jusnbgdn67uboqysujuz8pcaagbhnra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partaw | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 151df8..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryt3whrfmju4wgiks4g6npdimjgc1t8r34hi1thwhzbgfk63khiu1hnra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partax | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 031571..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryzajjabd5kfkt8e8qnck79zyy6ubasx5e7dm6mrfj9dxxxah9nd6anra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partay | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address d1f02b..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyrysipgwng3ixg8w4187qje9965bgmtzcsqmrjfapskg3jerxzj4dianra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partaz | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 695ade..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyrytjhmbh4hqm8e9bwa6a53gwfxyn7x63rytd9rr8ezth775cjd4tgynra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partba | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address a97e2e..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryokmdnd4tbb89b3w7f84k7i585b7aqbyrg38uiwaph7wejdf5eacwnra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partbb | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 631294..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryowi35tg968qyocj4c9rhxgxkggx8mydczokb18hw46nwym9xe4oenra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partbc | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 322a66..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryuowwi8hbhg8yxsthwzfga7ntkabgskhbwr6k7orr9eapfc9n4quonra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partbd | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address d30061..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyry1j1gghsby38cj677379g5diupwdxtmqbf5o9p96ohx5d77u3o4denra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partbe | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address c7f919..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryoougzsekwsgkhw3sndwu6zwny9k9gdj1t4sst793iws57jtixcncnra?v=huxwjuk87gg7gj13xxyae9unhzt3ej8xc5eap6t5phmadbjk3dxho"
+---+--------------------+--------------------------------------------------------------------+
| + | neal.tar.gz.partbf | safe://hyryyyym5z1c4mjed8ouqgq4cprrmbwkc6fwf5bj7rw14fcxibazqdh6bkc |
+---+--------------------+--------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyryubp7jh4z1hwjfrbjcsygx4dyto31er77cbtr79inkqkbzmz4oz4onra?v=h346sfgaeioocghm9zh95k5qc7cq3ba1brq9owbb99x4ddntyycgo"
+---+--------------------+--------------------------------------------------------------------+
| + | neal.tar.gz.partbg | safe://hyryyyyjpgodfqp1xbjg1wdyybg4kbn8kfx64x3u1pyrf3cqxjjimxtz6na |
+---+--------------------+--------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyry1js3gndkpyjptqetsdjj35bxk6d6mcwa3qsyda3d9sb1wop8e79onra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+--------------------+--------------------------------------------------------------------------------------------------+
| E | neal.tar.gz.partbh | <ClientError: Timeout after 90s when awaiting command ACK from Elders for data address 719139..> |
+---+--------------------+--------------------------------------------------------------------------------------------------+
FilesContainer created at: "safe://hyryyrysubjxxnub93m5iz3646t7rs7udfk99abp5shnocookakp7drtiienra?v=ha9d1dchnso3dtxuo39czxbky3t693sk61as3nyxhb7dyf9yox35o"
+---+--------------------+--------------------------------------------------------------------+
| + | neal.tar.gz.partbi | safe://hyryyyypwimkzg4zp4hq58sui1fdgh9hgu5hh1i9ckdc39xpq7f6ddsmbrr |
+---+--------------------+--------------------------------------------------------------------+

How full are the nodes now?

4 Likes

my uploads are still going fine with no errors using @TylerAbeoJordan script

7 Likes

Here is a container with 100 x 1MB files:-

“safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”

To download the whole lot do:- safe files get “safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”

Then if you want to check the validity of the download you can do this:-

Generate a file containing the md5sums of the downloaded files:- cd into the directory

md5sum * >> md5sums_downloaded

Then see if the md5sums are the same between what I uploaded and what you downloaded:-
diff md5sums md5sums_downloaded

The command prompt should return after printing nothing. If there are any differences in the md5sums for a file you’ll see output with the name of the file.

3 Likes

The network is saying , “hey, wait, timeout, stop being greedy!” :joy:

5 Likes

Here is a container with 1000 x 1MB files:-

“safe://hyryyrytbdqndesjq5f89cpgp5semm4hisignms1aumgmzdrdgi4i8be9khnra?v=hjjkc8ucfjta437xnozku7g9xf9fesyxdwwhbk3ha9tptggd3xygy”

To download the whole lot do:- safe files get “safe://hyryyrytbdqndesjq5f89cpgp5semm4hisignms1aumgmzdrdgi4i8be9khnra?v=hjjkc8ucfjta437xnozku7g9xf9fesyxdwwhbk3ha9tptggd3xygy”

Then if you want to check the validity of the download you can do this:-

cd into the directory

Generate a file containing the md5sums of the downloaded files:-
md5sum * >> md5sums_downloaded

Then see if the md5sums are the same between what I uploaded and what you downloaded:-
diff md5sums md5sums_downloaded

The command prompt should return after printing nothing if the md5sums of the downloaded files match the files I uploaded. If there are any differences in the md5sums for a file you’ll see output with the name of the file.

6 Likes

I’ve been thinking that perhaps the timeouts are a combination of distance to node and node activity.

Some people are getting no errors, so if they are closer to the nodes then maybe they are getting preferential access? I’m in Australia, so guessing physical distance is an issue.

Or alternatively perhaps it’s because I have a slow/limited upload speed - only about 500-600kb/s ? Maybe this is more likely, but a combo of resistance points seems credible too.

6 Likes

iv been hammering your script all night and still going strong I haven’t seen any errors from my quick glances at what’s going on.

would it possible to add an error counter to the script ?

3 Likes