That kind of worked, but I got another error:
🔗 Connected to the Network
Storing file "NNNN.jpg" of 4165873 bytes..
Did not store file "NNNN.jpg" to all nodes in the close group! Network Error Outbound Error.
1 Like
#WorksForMe - mostly…
Successfully stored file "build.sh" to ffa702b7d8e5fc4924048c8173f1b74bd9ceeb2a7dd1f6905099c3a478b63182
Storing file "scotcoin.png" of 4538 bytes..
Did not store file "scotcoin.png" to all nodes in the close group! Network Error Record not put to network properly.
Storing file ".gitignore" of 22 bytes..
Successfully stored file ".gitignore" to 8a794264ebcd148e2b08c5091916fd86e2797ab22c9206c646fde011e049745c
Storing file "README.md" of 478 bytes..
Successfully stored file "README.md" to 6b105be9ab38476dc79f6280a0f329b50f24603f4e401374ee9adbaeea54cb49
Storing file "pom.xml" of 5091 bytes..
Successfully stored file "pom.xml" to dd9e5e6beea42eb267230225ff8baf405772a6eab61fc8b58312da044ad772ed
Storing file "scotcoin.bom" of 2226 bytes..
Successfully stored file "scotcoin.bom" to 2666dd20e19f197bc42c697aef7c525fd7caf535427c506a80d5b6188ccbe1a3
Can the OP get changed to reflect this working SAFE_PEER?
EDIT: consistent failure on larger files
illie@gagarin:~/.local/share/safe/client/uploaded_files$ safe files upload ~/Videos/cooking/madhur/
Built with git version: 95b8dad / main / 95b8dad
Instantiating a SAFE client...
⠂ Connecting to The SAFE Network... The client still does not know enough network nodes.
🔗 Connected to the Network Storing file "madhur05.avi" of 362742180 bytes..
Did not store file "madhur05.avi" to all nodes in the close group! Network Error Record not put to network properly.
Storing file "madhur.jaffrey's.flavours.of.india.episode.04.avi" of 361880676 bytes..
Did not store file "madhur.jaffrey's.flavours.of.india.episode.04.avi" to all nodes in the close group! Network Error Record not put to network properly.
Storing file "madhur.jaffrey's.flavours.of.india.episode.02.avi" of 356676884 bytes..
Did not store file "madhur.jaffrey's.flavours.of.india.episode.02.avi" to all nodes in the close group! Network Error Record not put to network properly.
Storing file "madhur.jaffrey's.flavours.of.india.episode.06.avi" of 357022232 bytes..
Did not store file "madhur.jaffrey's.flavours.of.india.episode.06.avi" to all nodes in the close group! Network Error Record not put to network properly.
Storing file "madhur.jaffrey's.flavours.of.india.episode.03.avi" of 360133404 bytes..
Did not store file "madhur.jaffrey's.flavours.of.india.episode.03.avi" to all nodes in the close group! Network Error Record not put to network properly.
Storing file "madhur.jaffrey's.flavours.of.india.episode.01.avi" of 358941706 bytes..
Did not store file "madhur.jaffrey's.flavours.of.india.episode.01.avi" to all nodes in the close group! Network Error Record not put to network properly.
Writing 8 bytes to "/home/willie/.local/share/safe/client/uploaded_files/file_names_2023-07-09_19-33-24"
1 Like
@aatonnomicc
can you try to upload BegBlag again please?
I get a failure on this 15MB file and larger vids
smaller files <10Mb seem to be OK for me
1 Like
Have you retrieved any registers?
I have not really played with them, am I doing this right.
josh@evo:~/upload$ safe register get register
Built with git version: 95b8dad / main / 95b8dad
Instantiating a SAFE client...
⠄ Connecting to The SAFE Network... The client still does not know enough network nodes.
🔗 Connected to the Network Register name passed in via `register get` is 'register'...
Trying to retrieve Register from 7e06f7(01111110).., 3006
Did not retrieve Register 'register' from all nodes in the close group! Protocol error Register not found: RegisterAddress { name: 7e06f7(01111110).., tag: 3006 }.
2 Likes
I’ve noticed something and gained an insight into what is going on with RAM and records.
RAM usage is continuing to go up on my nodes on this AWS Instance and I’m continuing to lose nodes. I now only have 23 of the original 50 running on this 4GB AWS Instance. Their RAM usage is now 150MB to 160MB for most of the nodes. However, I spotted a few that are much lower. The one that is lowest is Node 49 and it’s only at 69MB. it’s holding 2048 records which I think must be a limit like it was 1024 for an earlier network. So I think it’s not storing any more data. So it seems that RAM consumption of a Node isn’t strictly related to amount of data stored or the RAM usage would have ballooned earlier when it was filled.
But it might be related to ongoing participation in record storing over time.
And it can’t be related to record retrieval as with its 2048 records it should be as busy as any other node.
A quick check of other nodes with markedly lower RAM consumption on this Instance shows that generally nodes with a higher number of records at the moment.
If I had the skillz I’d be have a deep dive into this and plotting it rather than just picking a few at the extremes to look at.
4 Likes
I moaned about registers a couple of days ago and IIRC @joshuef explained why they were unavailable for now.
Apologies to all, I have been distracted with RealLife and have not been able to give this testnet the full attention it deserves.
What worked - well kinda - before was
safe register create my_new_register ← create
safe register edit my_new_register "the first entry" ← add entries
then safe register get my_new_register ← this allegedly retreives your register and previously reported success. There was no way to actually view the contents of the retreived register but thats just a wee implementation detail that the team will sort out soon enough - said he hopefully.
However as you confirmed, we are not getting that far on this testnet.
2 Likes
Could be this
3 Likes
Tell me that 1024 chunk limit will be greatly increased soon – please!!!
1 Like
I think it is 2048 now
2 Likes
Better - but I would have thought each node should be capable of storing tera/peta bytes of chunks.
1 Like
Although I think so because I thought I read it in the OP and it is not there so, perhaps a hallucination.
2 Likes
The 1024 limit on chunks also applied to no of entries in each register.
I had a wee script that continually added entries to a register. It broke when I exceeded 1024 loops.
1 Like
Not totally senile yet, read 2048 records here.
2 Likes
This is a thing we are working on. In my view all ants are kinda equal. If we want more data, we have more ants. That way we grow the network with large numbers of nodes (more security). It also means when a huge node goes down it takes out data across the network and not concentrated data in one section/group.
i.e. the new large nodes are late computers running lots of nodes. It makes a lot of sense and keeps nodes equalised, whilst allowing folk to run massive computers with lots of node instances on them.
13 Likes
OK that makes sense.
But I still might want more than 1024/2048 items in my list/register.
5 Likes
Yes, what we need there is a link entry. That last entry will point to the next register. It’s a bit messy, though and we need to deep dive it. What I mean is a long long register, and if you want to read the last entry you need to get all registers. So we need a clever route there.
6 Likes
Oddly though some nodes have surpassed this!
12D3KooWFCWkq683wS9FvNgSqXxtR3VtEifeDuEkzGmLfNeGaUfT 2832 302.305 0.0 2210
I’m not sure if this is our deletion code failing (ie, the node still actually only has 2048 its aware of), or what.
We’ll be digging in there!
2 Likes
Op updated for export SAFE_PEERS="/ip4/54.82.83.176/tcp/35175/p2p/12D3KooWPhVptRnwNXfqtENhq1GNxCnyNBPcJu7xALPFQ72KTBii"
2 Likes
Potentially, it’s a holistic thing to consider. For now if you want more capacity you’ll have to add more nodes!
@roland can you take a look at this?
@Vort can you provide some logs for this time period? We have seen some small loops around data replication, but steps were taken to prevent this. This sounds like that’s not working though. (cc @qi_ma )
3 Likes
grep -c Replicate /d/snlogs2/*
/d/snlogs2/safenode.log.20230709T190904:741
/d/snlogs2/safenode.log.20230709T192100:866
/d/snlogs2/safenode.log.20230709T193349:695
/d/snlogs2/safenode.log.20230709T194753:832
/d/snlogs2/safenode.log.20230709T200033:749
/d/snlogs2/safenode.log.20230709T200611:1182
/d/snlogs2/safenode.log.20230709T200922:1309
/d/snlogs2/safenode.log.20230709T201240:1420
/d/snlogs2/safenode.log.20230709T201545:1392
/d/snlogs2/safenode.log.20230709T201903:1414
/d/snlogs2/safenode.log.20230709T202243:1391
/d/snlogs2/safenode.log.20230709T202605:1439
/d/snlogs2/safenode.log.20230709T202830:1401
/d/snlogs2/safenode.log.20230709T203046:1428
/d/snlogs2/safenode.log.20230709T203433:1386
/d/snlogs2/safenode.log.20230709T203722:1473
/d/snlogs2/safenode.log.20230709T204312:1324
/d/snlogs2/safenode.log.20230709T204613:1446
/d/snlogs2/safenode.log.20230709T205044:1310
/d/snlogs2/safenode.log.20230709T205235:1473
/d/snlogs2/safenode.log.20230709T205418:1420
/d/snlogs2/safenode.log.20230709T205649:1403
/d/snlogs2/safenode.log.20230709T205842:1468
/d/snlogs2/safenode.log.20230709T210050:1450
/d/snlogs2/safenode.log.20230709T210243:1610
/d/snlogs2/safenode.log.20230709T210516:1385
/d/snlogs2/safenode.log.20230709T210713:1465
/d/snlogs2/safenode.log.20230709T211054:1288
/d/snlogs2/safenode.log.20230709T211329:1542
/d/snlogs2/safenode.log.20230709T211608:1444
/d/snlogs2/safenode.log.20230709T211911:1444
/d/snlogs2/safenode.log.20230709T212251:1323
/d/snlogs2/safenode.log.20230709T212540:1362
/d/snlogs2/safenode.log.20230709T212823:1417
/d/snlogs2/safenode.log.20230709T213206:1344
/d/snlogs2/safenode.log.20230709T213451:1417
/d/snlogs2/safenode.log.20230709T213823:1327
/d/snlogs2/safenode.log.20230709T214222:1373
/d/snlogs2/safenode.log.20230709T214535:1399
/d/snlogs2/safenode.log.20230709T214848:1436
/d/snlogs2/safenode.log.20230709T215133:1445
/d/snlogs2/safenode.log.20230709T220115:925
/d/snlogs2/safenode.log.20230709T221415:720
/d/snlogs2/safenode.log.20230709T222820:736
/d/snlogs2/safenode.log.20230709T224207:848
/d/snlogs2/safenode.log.20230709T225348:814
Probably problem started to appear at ~20230709T200611 and stopped at ~20230709T220115.
5 Likes