So would this be correct for me to have a setup, that is otherwise similar, but correctly running Vdash?
nohup SN_LOG=all safenode --port=12000 &
nohup SN_LOG=all safenode --port=12001 &
And why in the previous testnets the PUT record in Vdash was, if not correct, not so wildly off…
Maybe you had SN_LOG set in your shell environment. Or maybe maidsafe altered the logging level for that message. 
That’s it I suspect. I haven’t used SN_LOG=all for a while now
Repeating over and over again also ends up uploading the file. With each repetition, fewer chunks remain unraised.
I’m uploading big file, having to retry constantly, and got a new error now:
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 1780 chunks
Will now attempt to upload them...
Error:
0: Failed to pay for chunks: Transfer Error Failed to send tokens due to The storage payment transfer was not successfully registered in the network: CouldNotSendMoney("Network Error Could not retrieve the record after storing it: dc0322c1fa7845829621f6cbfcf5c9245ee60b6c1a4d3f30806106b1ac421db8(f415db8dfcb075627becf994dfbdf2d45cc6861d0be4d589c03a41e14c8bcf23).").
EDIT: Deleting the wallet and re-fauceting solved the problem. 
you can have a try with batch_size reduced to say 16 or even 8.
btw. I was testing with the “ubuntu-18.06-desktop-amd64.iso”, which will be split into 4798 chunks.
not aware just two minor version diff will have so large size diff.
They don’t. Mine is about the same, but the upload progress has been going like this (Edited the peers away for clarity):
time safe files upload ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Chunking 1 files...
Input was split into 4058 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: 022273a4521a29a03af2889940f46e660cde6425d80c8fbe038ede2902c88ee3(c73f871b429a5a376555754663dda49771edd4a724a1be58b277a1842c7128c3).
1: Could not retrieve the record after storing it: 022273a4521a29a03af2889940f46e660cde6425d80c8fbe038ede2902c88ee3(c73f871b429a5a376555754663dda49771edd4a724a1be58b277a1842c7128c3)
time safe files upload ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 3895 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: 14a4548e9a7adbb92ca0e3d932e9e627686affd7de80ef2b1ccb2b06c9454a85(4ad61e7b865bba6a5e7378e70fbac5435aa0d1973ab720642cac4772fc135ff2).
1: Could not retrieve the record after storing it: 14a4548e9a7adbb92ca0e3d932e9e627686affd7de80ef2b1ccb2b06c9454a85(4ad61e7b865bba6a5e7378e70fbac5435aa0d1973ab720642cac4772fc135ff2)
time safe files upload --batch-size 40 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 3535 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: 3c8f55ba6fffdd50483937a9ac75aaaa03f59e9e62d5c01fca588472fa712b29(615709c901b360541db9a13c817b57ae01e17688dff7d784a7ae8e6925f45b9d).
1: Could not retrieve the record after storing it: 3c8f55ba6fffdd50483937a9ac75aaaa03f59e9e62d5c01fca588472fa712b29(615709c901b360541db9a13c817b57ae01e17688dff7d784a7ae8e6925f45b9d)
time safe files upload --batch-size 40 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 3088 chunks
Will now attempt to upload them...
⠤ [00:00:12] [----------------------------------------] 0/3088 ^C
CTRL+C
time safe files upload --batch-size 30 ubuntu-18.04.4-desktop-amd64.iso
Error:
0: Transfer Error I/O error: Could not read and deserialize wallet file after multiple attempts.
1: I/O error: Could not read and deserialize wallet file after multiple attempts
2: Could not read and deserialize wallet file after multiple attempts
time safe files upload --batch-size 30 ubuntu-18.04.4-desktop-amd64.iso
0: Transfer Error I/O error: Could not read and deserialize wallet file after multiple attempts.
1: I/O error: Could not read and deserialize wallet file after multiple attempts
2: Could not read and deserialize wallet file after multiple attempts
time safe files upload --batch-size 30 ubuntu-18.04.4-desktop-amd64.iso
0: Transfer Error I/O error: Could not read and deserialize wallet file after multiple attempts.
1: I/O error: Could not read and deserialize wallet file after multiple attempts
2: Could not read and deserialize wallet file after multiple attempts
DELETED THE WALLET
safe wallet get-faucet 178.128.33.82:8000
Requesting token for wallet address: 969227f1c468ecc0d135f2ca1ad118d4f2615c93ecc18f67ce121c1af30f981dafcb88eec8b5bc1bd8379fc6f264f4f8...
Successfully parsed transfer.
Verifying transfer with the Network...
Successfully verified transfer.
Successfully stored cash_note to wallet dir.
Old balance: 0.000000000
New balance: 100.000000000
Successfully got tokens from faucet.
time safe files upload --batch-size 30 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 3088 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: 6f7dc22c67f5c5515ce1863a4a4328cf1a1ba1139ae27cffe5b1a6174a7192b4(13c80e5a5dcb809cbcae2b3d86c036d89e76a12baa0ce412ce3156a27f109148).
1: Could not retrieve the record after storing it: 6f7dc22c67f5c5515ce1863a4a4328cf1a1ba1139ae27cffe5b1a6174a7192b4(13c80e5a5dcb809cbcae2b3d86c036d89e76a12baa0ce412ce3156a27f109148)
time safe files upload --batch-size 20 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 2249 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: 80ec767a1056563b12de4172b45a6720ee090143afca779981810e19031b94b1(30353b4a030162313788e22bb6eeec1f71d085ef27efb2b888cdcce3098e83dd).
1: Could not retrieve the record after storing it: 80ec767a1056563b12de4172b45a6720ee090143afca779981810e19031b94b1(30353b4a030162313788e22bb6eeec1f71d085ef27efb2b888cdcce3098e83dd)
time safe files upload --batch-size 10 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 2031 chunks
Will now attempt to upload them...
⠈ [00:11:48] [#####>----------------------------------] 260/2031
CTRL+C
time safe files upload --batch-size 15 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 1780 chunks
Will now attempt to upload them...
Error:
0: Failed to pay for chunks: Transfer Error Failed to send tokens due to The storage payment transfer was not successfully registered in the network: CouldNotSendMoney("Network Error Could not retrieve the record after storing it: 2d904ab51d3dc7a92ed9ef8957b7b59fb6be588a0743dcae28e134d3fbdd5443(67655160322f412d9b5ed391c2f9a46b7dec5aa0fa74253b15ec735a0e28faa5).").
time safe files upload --batch-size 15 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 1780 chunks
Will now attempt to upload them...
Error:
0: Failed to pay for chunks: Transfer Error Failed to send tokens due to The storage payment transfer was not successfully registered in the network: CouldNotSendMoney("Network Error Could not retrieve the record after storing it: dc0322c1fa7845829621f6cbfcf5c9245ee60b6c1a4d3f30806106b1ac421db8(f415db8dfcb075627becf994dfbdf2d45cc6861d0be4d589c03a41e14c8bcf23).").
time safe files upload --batch-size 10 ubuntu-18.04.4-desktop-amd64.iso
Starting to chunk "ubuntu-18.04.4-desktop-amd64.iso" now.
Input was split into 1780 chunks
Will now attempt to upload them...
Error:
0: Failed to pay for chunks: Transfer Error Failed to send tokens due to The storage payment transfer was not successfully registered in the network: CouldNotSendMoney("Network Error Could not retrieve the record after storing it: 878615ebe82a6447e56b8f876ef3fd21f00af54a445f100d61588dd9d43e38f8(74dc65592fada47593e1a96e0ebc460504c636f48d61900ca4ba9937dbf0d02d).").
ah, I see.
right, the uploaded chunks will be filtered out.
Yeah, it’s just a bit misleading to say that “input was split” into something, when in reality… what? Are the pre-chunked chunks saved somewhere and it is just retrying those? Or is it really re-chunking the whole file, but tells us about only the chunks that have not been uploaded?
Memory was so flat it was boring last net.
We are climbing this time. @aatonnomicc seems more so than me but he started earlier than me.
Upload from home fail
Batch size 5
🔗 Connected to the Network Starting to chunk "/home/willie/Music/Bowie/" now.
Chunking 11 files...
Input was split into 187 chunks
Will now attempt to upload them...
Error:
0: Network Error Could not retrieve the record after storing it: e5b5341d62ceaa4d1cedba2bc3cd6c6bc8db9f0c9e6537c9d48014aad19bca4e(6c38678d9fae3e8b4013f1abcec932aa0e5bbf7f16d70f8715ea9d501cf1cdc2).
1: Could not retrieve the record after storing it: e5b5341d62ceaa4d1cedba2bc3cd6c6bc8db9f0c9e6537c9d48014aad19bca4e(6c38678d9fae3e8b4013f1abcec932aa0e5bbf7f16d70f8715ea9d501cf1cdc2)
Location:
sn_cli/src/subcommands/files/mod.rs:424
some times @aatonnomicc is the problem
I restarted a few vps about that time and went form 50 nodes to 75 on 5 vps and shortly after went from 50 to 90 nodes on another 5 vps’s.
thought memory usage was going so well i could push the boat out a little
while i was trying to figure out how many nodes i could fit on a machine it struck me that if people were running more nodes than they had hard disk space for there could be a cascade failure if the network filled up and sudden the nodes that were not full could not store any more data.
is there a way for the nodes to reserve there maximum capacity on start up to avoid that situation ?
Maybe 
but it is trending up for me too, so idk. Lets see if it levels out. Hope so some are steep.
Or are you saying that you (potentially) caused network wide chaos? 
Already seems better for me, although 3 are reaching for the stars.
I have tried batch sizes 16,32,64 and 128 and the Could not retrieve the record after storing it looks like a function of the chunk size. I get it roughly after 3xbatch-size chunks. I have tried total 24 runs with the 4 different chunk sizes and the error always came after number of chunks between 2-4x batch-size.
Not sure but it was going perfectly than I did that and mem usage started going up so cards on the table it was suspicious timing
From now on once I have nodes up I’ll leave them alone till the end as we have a young hatchling network
Your hanging round with you know who to much 
I resemble that remark.
Anymore from you and I may do a deep dive on the failings of the team that couldny play for toffee
Uploading from home works for me, although I did have to retry each file twice (default chunk size).
“The Teskey Brothers - Rain - KXT Live Sessions.webm” e54146f527c31b09f41542dd937b5a317b027048af6c75b817666875270c8eb6
“The Teskey Brothers - Hold Me (Official Video).webm” 2edd216e7b939080588c13815444bf9e472bcfad6137a242c7ba2f26f8bc9ef1
I only got the payment message on the second try, and seems like I’m paying for uploading only the last few chunks that failed initially?
**************************************
* Uploaded Files *
**************************************
"The Teskey Brothers - Rain - KXT Live Sessions.webm" e54146f527c31b09f41542dd937b5a317b027048af6c75b817666875270c8eb6
Uploaded 2 chunks (with 2 exist chunks) in 39 seconds
**************************************
* Payment Details *
**************************************
Made payment of 0.000000000 for 2 chunks
Made payment of 0.000000000 for royalties fees
New wallet balance: 99.999992260
/me grabs - cos I missed them the last time they toured
I am sitting back on this one tonight - busy with other stuff,so Im just keeping an eye on the amateur hammer weilders …