[Offline] 50GB Static Network - Round 3

Development Updates
62

hmm. I think this was a failure to upload then. I listed the contents of the container and it’s just not there:-

safe files ls "safe://hyryyryubtjfaokg9ag58a1j3jdxe4hutcehcrdikygketfmhkw7rcjptuynra?v=hhg7m7s4n9qibzce5oyidzsykpq9mqokczcr9db8o61m8i4drum6o" | grep 2MB_70
| 1048576 | 1673069149 | 1673069149 | 2MB_70   |
| 1048576 | 1673069149 | 1673069149 | 2MB_700  |
| 1048576 | 1673069149 | 1673069149 | 2MB_702  |
| 1048576 | 1673069149 | 1673069149 | 2MB_703  |
| 1048576 | 1673069149 | 1673069149 | 2MB_704  |
| 1048576 | 1673069149 | 1673069149 | 2MB_705  |
| 1048576 | 1673069149 | 1673069149 | 2MB_706  |
| 1048576 | 1673069149 | 1673069149 | 2MB_707  |
| 1048576 | 1673069149 | 1673069149 | 2MB_708  |
| 1048576 | 1673069149 | 1673069149 | 2MB_709  |

I have to admit I didn’t check the output of the upload command so I don’t know if there was an error.

I think I’ll have to start checking that in future.

The file is definitely there in the directory where I sent it from and that’s shown by the md5sum file as well.

So there is an issue for you @joshuef but I don’t have any error or logging for you to get a grip on it with. :frowning:

2 Likes
63

How big is the file?

Could It have been a connect or timeout iissue?

2 Likes
64

why is upload speed so slow in my case? I try to upload about 9giga, but about 6hours I spend on and not finished yet. My speed upload its about 22MB/s to normal centralized storage. Its because there is only(?) one node?

65

Sounds normal to me. Uploading a 1.2GB directory took well over an hour and my upload speed is rated at 20MB/s. I’d expect 9GB to take 10-12 hours.

66

It was exactly 2MB as were all the files. I wonder what the behaviour is if there is a momentary disconnect?

I really wish I’d paid more attention to the output. I’m going to start saving it to a file from now on.

2 Likes
67

Okay, well something to retry and keep an eye on at least. I haven’t come across this bug in other testing, so lets see what we can nail down there going forwards.

FYIs: metricbeat still happy, tast-data still available

node-16: 33G    total
node-13: 33G    total
node-11: 33G    total
node-1: 33G     total
node-18: 33G    total
node-15: 33G    total
node-12: 33G    total
node-24: 33G    total
node-10: 54G    total
node-2: 33G     total
node-23: 50G    total
node-3: 33G     total
node-22: 50G    total
node-d: 41G     total
node-14: 42G    total
node-20: 33G    total
node-6: 41G     total
node-19: 55G    total
node-9: 49G     total
node-25: 52G    total
node-4: 50G     total
node-8: 50G     total
node-21: 50G    total
node-5: 49G     total
node-17: 41G    total
node-19: 55G    total
node-1: 33G     total
12 Likes
68

alba gu brath!!!

3 Likes
69

Another cock and bull story

FilesContainer created at: "safe://hyryyryif9cds95t7by8f57ajziox9d6npouzdshshsjq3wbhms4fnd46grnra?v=hdx8wma49d1utci1emchhmzejrcsjym6uxt5jzf3aiggqo4ijdkso"
+---+-----------------------------+--------------------------------------------------------------------+
| + | lecocketpierreshistoire.txt | safe://hy8oycyeafkibtjrrh63cpryam9buhc5hmmjmoewm94xnpwco4dzhxkui5c |
+---+-----------------------------+--------------------------------------------------------------------+
70

I think the upload speed has really slowed. I’ve been uploading another 2000 2MB files for about 3 hours now and it didn’t take this long before.

1 Like
71

whit?! node-d

3 Likes
72

Copy/paste gremlins? There are also two 1’s no 7 and the d.

3 Likes
73

I’m using @TylerAbeoJordan 's ‘isu’ upload, download and verify script and have an error straight away on the third file:-

isu>> FilesContainer created at: “safe://hyryyrywic4nuqo5rtbi6ftide9wpxdhak8d9bnisbua1unbj7p76ktjuoenra?v=hpcq7thet67kftnt4sjjodt155oprmapxjx4rj16kq79byqnk9u6o”

±–±-----------------±-------------------------------------------------------------------+

| + | c833234072aba77d | safe://hyryyyyp1aj7mkb58nkatbj78chbw94h5hqehmkby7u7m1h7wbuk7c98kpr |

±–±-----------------±-------------------------------------------------------------------+

isu>> Error Ratio:/2 (errors per total uploads) – total bytes uploaded:9309182

isu>> uploading filename:f5b1c21349428cd2 – size in bytes:4930201; please wait …

isu>> file: c833234072aba77d has good checksum

Error:

0: ClientError: Error received from the network: DataNotFound(Register(RegisterAddress { name: e655dd(11100110)…, tag: 1100 }))

1: Error received from the network: DataNotFound(Register(RegisterAddress { name: e655dd(11100110)…, tag: 1100 }))

2: Requested data not found: Register(RegisterAddress { name: e655dd(11100110)…, tag: 1100 })

Location:

sn_cli/src/subcommands/files.rs:213

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.

Run with RUST_BACKTRACE=full to include source snippets.

isu>>

isu>> Error Ratio:/3 (errors per total uploads) – total bytes uploaded:14239383

Error:

0: InvalidXorUrl: Problem parsing the URL “safe://”: missing name

Location:

sn_cli/src/subcommands/helpers.rs:273

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.

Run with RUST_BACKTRACE=full to include source snippets.

isu>> file: f5b1c21349428cd2 has BAD checksum

isu>> uploading filename:193077b7d0fceb5e – size in bytes:8697087; please wait …

isu>> Error during upload, retrying; please wait …

isu>> Error during upload, retrying; please wait …

isu>> Error during upload, retrying; please wait …

5 Likes
74

Something similar

willie@gagarin:~$ jobs
[1]   Running                 /home/willie/.safe/cli/safe files put -r /fgfs/Aircraft/c182s/ &
[2]   Running                 /home/willie/.safe/cli/safe files put -r /fgfs/Aircraft/737-800YV/ &
[3]   Running                 /home/willie/.safe/cli/safe files put -r /fgfs/Aircraft/f16 &
[7]-  Running                 /home/willie/.safe/cli/safe files put -r /fgfs/Aircraft/c172p-detailed/ &
[8]+  Running                 /home/willie/.safe/cli/safe files put -r /fgfs/Scenery/Scotland/ &
willie@gagarin:~$ Error: 
   0: ClientError: Error received from the network: DataNotFound(Bytes(ChunkAddress(fee026(11111110)..)))
   1: Error received from the network: DataNotFound(Bytes(ChunkAddress(fee026(11111110)..)))
   2: Requested data not found: Bytes(ChunkAddress(fee026(11111110)..))

Location:
   sn_cli/src/subcommands/files.rs:213

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

[1]   Exit 1                  /home/willie/.safe/cli/safe files put -r /fgfs/Aircraft/c182s/
2 Likes
75

think we just ran out of space!!

image
image1920×1039 273 KB

7 Likes
76

There is still room for very small files :slight_smile:

Note the variety of error msgs and formats though

willie@gagarin:~/tmp/threads$ ll
total 16
-rw-rw-r-- 1 willie willie  151 Jan  8 01:18 Cargo.lock
-rw-rw-r-- 1 willie willie  176 Jan  8 01:17 Cargo.toml
drwxrwxr-x 2 willie willie 4096 Jan  8 01:31 src
drwxrwxr-x 4 willie willie 4096 Jan  8 01:21 target
willie@gagarin:~/tmp/threads$ safe files put Cargo.toml 
Error: 
   0: ClientError: Error received from the network: DataNotFound(Bytes(ChunkAddress(994cf1(10011001)..)))
   1: Error received from the network: DataNotFound(Bytes(ChunkAddress(994cf1(10011001)..)))
   2: Requested data not found: Bytes(ChunkAddress(994cf1(10011001)..))

Location:
   sn_cli/src/subcommands/files.rs:213

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
willie@gagarin:~/tmp/threads$ safe files put Cargo.toml 
FilesContainer created at: "safe://hyryyryta8ckhuhqy98kbx8f6wmfr4shh5wrgpxwu6bgn64xeeymxddiykonra?v=hwwti6tnqpo8p5iik6z45xnph77rqf6xmoq6zuo5fucitfh7kwx9o"
+---+------------+--------------------------------------------------------------------+
| + | Cargo.toml | safe://hyryyyyk4ho9szcasorf7tpoy4q4q16rqyc6cuqfaaoufkcxpuc86wr7x6o |
+---+------------+--------------------------------------------------------------------+
willie@gagarin:~/tmp/threads$ safe files put Cargo.lock
FilesContainer created at: "safe://hyryyryi31m6cdb9nmedwcnnfjbqjsztbh5stqzqzpnzqam1pr3ji51qryonra?v=hnrijhg7qx435iejzzjm1i5oc3qgw3s3b36ej9r7wwe8nmgxjnfey"
+---+------------+-------------------------------------------------------------------------------------------------------+
| E | Cargo.lock | <ClientError: Error received from the network: DataNotFound(Bytes(ChunkAddress(f55a64(11110101)..)))> |
+---+------------+-------------------------------------------------------------------------------------------------------+

I think the testnet has done its job and given the devs valuable results.

Onwards and upwards!

6 Likes
77

I don’t think I’ve read a test net thread as positive as this one. Is there something more basic about it?

7 Likes
78

Same here, get still working fine but no put.
It worked great till we hit 50gb, but i doubt we got anywhere close to 100gb on all nodes.
Not sure whether this condition was fully satisfied

Logs will tell I suppose…

3 Likes
79

So is this a potential attack vector that may need addressing in the future?

The attack being pay for say 1TB of data, then the attackers keeps uploading it from as many places as possible to increase their upload bandwidth. This is effectively a DDOS attack on the sections holding the data and more specifically the nodes holding the data (and perhaps the elders of those sections)

7 Likes
80

Perhaps there is a way to limit the number of upload streams/connections per client ID (w/payment receipt)?

2 Likes
81

I think even gets of previously successful puts are breaking now. I can’t download some of the directories with large numbers of files in them. Even the ‘ls’ is failing.

I can still ls and download the one that had the missing file ‘2MB_701’:-

"safe://hyryyryubtjfaokg9ag58a1j3jdxe4hutcehcrdikygketfmhkw7rcjptuynra?v=hhg7m7s4n9qibzce5oyidzsykpq9mqokczcr9db8o61m8i4drum6o"

But these ones that I put earlier than that:-

“safe://hyryyrytbdqndesjq5f89cpgp5semm4hisignms1aumgmzdrdgi4i8be9khnra?v=hjjkc8ucfjta437xnozku7g9xf9fesyxdwwhbk3ha9tptggd3xygy”

and

“safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”

are failing with this error:-

time safe files ls “safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”
Error: 
   0: UrlError: InvalidXorUrl: Problem parsing the URL "“safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”": relative URL without a base
   1: InvalidXorUrl: Problem parsing the URL "“safe://hyryyry1zbn9o5xig9x1ddp5usnay1qtxni79noumpbg87fgxg9n4b35j9hnra?v=h6ui98owiuumijhhuafi4nkxi4qbn83d8k1ktrqi4kkycy9nwwsoy”": relative URL without a base

Location:
   sn_cli/src/subcommands/files.rs:355

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

real	0m0.088s
user	0m0.030s
sys	0m0.004s
3 Likes