Do some of the above keywords also require TRACE to be on?
I haven’t spotted the keyword ‘Chunk’ or ‘replication’ yet at the INFO+ level.
3 Likes
Yes, they are all in TRACE level.
And, right, it shall be mentioned in the OP.
Thx for the remind.
5 Likes
I uploaded about 600MB file, and tried to download it. Unfortunately I only had a work project at hand, that I’m not willing to share publicly. Result:
Chunks error Not all chunks were retrieved, expected 1243, retrieved 1238
Five chunks short. I’m going to give another try soon.
Also, the safe files download tried to download files that I uploaded in previous testnets too. I guess I should have cleared some folder somewhere?
1 Like
Good idea. Will try to do the same.
What I noticed already - RAM usage is 1.4 GB.
I hope this is not a limitation of protocol, but client is just not optimized yet.
Otherwise it will be not possible to upload files larger than available RAM amount.
1 Like
It’s not a protocol limitation as that would be 
2 Likes
Yes it will try to download any files recorded in the uploaded_files folder.
Worth to clean up that folder (or just delete the old file_names files).
2 Likes
This time success!
6 Likes
My test upload of 0.6 GB file took 36 minutes. Process used 4.9 GB of network traffic (looks like 8x replication).
Storing file "CC-01.iso" of 662310912 bytes..
Successfully stored file "CC-01.iso" to 15d9243ab2cb55e14a573e3db624ca4082248e9883924446191a34913568eda6
Will try to download it next.
upd. Download is finished too.
Resource usage: 20 minutes, 0.7 GB of RAM (with spike to 2.3 GB at the end) and 6.2 GB of traffic.
10x download size is too much to be honest.
4 Likes
For those running a node using SN_LOG=all I have updated vdash to display some of the system and process metrics recently added to the logfile (by the sn_metrics crate).
To update vdash to v0.8.3 and run it:
cargo install vdash
vdash /tmp/safenode/safenode.log
5 Likes
SN_LOG=all ./safenode --log-dir=~/replicationnet/safenode0 --root-dir=~/replicationnet/safenodedata0
The “~” as a user home directory does not work. A folder named “~” is created in current dir instead.
2 Likes
I noticed suspicious thing:
My node uptime is 3 hours.
Network for sure contains more than 1 GB of data.
But data directory for my node is empty, no chunks.
Is it supposed to be like that?
6 Likes
I think windows terminals would do that. We should remember that for future updates
4 Likes
Check this out, might help:
1 Like
I know where chunks should be.
Because 1. I set directory manually. 2. I noticed chunks during previous testnet there. 3. And it is not completely empty: there is empty directory record_store inside data dir, so path is correct for sure.
1 Like
With a network of 2000+ nodes, seems possible some may be empty this early, no?
4 Likes
1 GB+ of pure data means ~10 GB of replicated data.
If chunk size is 1 MB, then there should be 10000 of them.
It is possible to be that “unlucky” of course, but I doubt it.
4 Likes
If log is enabled, you can look for That's a store chunk in or Chunk received for replication to confirm whether there is chunk flow in.
1 Like
Logs are enabled. But only INFO and higher. Did not knew that TRACE should be enabled manually.
And I see no chunk messages in my logs. Either they at TRACE or they just do not exists. Peer id is 12D3KooWQunT6gwcqQqE5qpLdZuSMhGVy6t564LPbNGDXGDRCU5B by the way.
1 Like
I drafted a script:
#!/bin/bash
echo "PeerAdded" `grep "PeerAdded:" "$1" | wc -l`
echo "PeerAdded (unique)" `grep -o "PeerAdded: [0-9a-zA-Z]*" "$1" | sort -u | wc -l`
echo "Detected dead peer" `grep "Detected dead peer" "$1" | wc -l`
echo "Sending a replication list" `grep "Sending a replication list" "$1" | wc -l`
echo "Replicate list received from" `grep "Replicate list received from" "$1" | wc -l`
echo "Fetching replication" `grep "Fetching replication" "$1" | wc -l`
echo "Replicating chunk" `grep "Replicating chunk" "$1" | wc -l`
echo "Chunk received for replication" `grep "Chunk received for replication" "$1" | wc -l`
and the results for first ~10 mins of running node:
$ ./stats.sh safenode0_data/safenode.log.20230607T152926
PeerAdded 100
PeerAdded (unique) 100
Detected dead peer 0
Sending a replication list 0
Replicate list received from 0
Fetching replication 0
Replicating chunk 0
Chunk received for replication 0
Results of $ grep "TRACE sn_logging::metrics" safenode0_data/safenode.log.20230607T152926 attached.
grep_trace_logging_metrics.log.zip (4.6 KB)
(EDIT) node was run using: SN_LOG=all ./safenode --log-dir=safenode0_data --root-dir=safenode0_data &
9 Likes
the mem & cpu usage is quite stable.
the total_bytes_written says 1.57MB written, however may include log writting and sending copy to network. .
Do you see any chunk files created under the storage folder ?
4 Likes