Right, cool, thanks.
Yeah, the auto restart has now been bumped up to a very high priority issue and will be dealt with before the beta launch. Thanks for the feedback.
Edit: in both the node manager and the launchpad.
Right, cool, thanks.
Yeah, the auto restart has now been bumped up to a very high priority issue and will be dealt with before the beta launch. Thanks for the feedback.
Edit: in both the node manager and the launchpad.
i think thatâs a very good time point to having removed the autostart - yes ⌠people will want to go up in node count then and test out their limits ⌠might well result in blocked systems otherwise âŚ
Actually, I just want to make sure weâre talking about the same thing here.
When I say âauto restartâ, what I mean is that the services will automatically start after the operating system has rebooted.
Do you mean the same thing? Or are you talking about the launchpad starting all the nodes after resource allocation?
exactly what iâm talking about - yes
ps: after reboot / user login (user services)
Has anyone had any success in starting a small no of nodes?
say 20 ish and then allocating more resources - hit O and change 100 to 150 to give 30 nodes?
I can get 20 nodes started OK but cannot add any more, not 10 not 4, not 2âŚ
Yep once again failing to start >20 nodes
2024-05-30T18:58:35.279533Z DEBUG sn_node_manager/src/cmd/node.rs:101: Parsing peers...
2024-05-30T18:58:35.279563Z INFO sn_peers_acquisition/src/lib.rs:88: Using peers supplied with the --peer argument(s) or SAFE_PEERS
2024-05-30T18:58:44.220073Z ERROR node-launchpad/src/components/home.rs:401: Error while maintaining 24 running nodes
0: Failed to add one or more services
Suggestion: However, any services that were successfully added will be usable.
2024-05-30T18:58:44.220122Z DEBUG node-launchpad/src/app.rs:139: HomeActions(ServiceManagerOperationCompleted)
2024-05-30T18:58:44.220307Z INFO node-launchpad/src/components/home.rs:82: Loaded node registry. Runnign nodes: 20
Remind me to submit a trivial PR to sort âRunnignâ - Its doing my spelling-nazi head inâŚ
EDIT: done PR #1795
Yes, I went from 1 to 5 to 10 in stages by changing resource allocation and using ctrl+g to start them
Let me know how you get on trying to get over 20
I just restarted there with 80Gb (16 nodes) and then added another 40GB to take me to a notional 24
Launchpad currently wonât show me any more than 20 nodes here.
Has anyone seen launchpad display >20 nodes?
EDIT: Yes @Josh see below
and in the launchpad logs there is little to be gleaned other than it now thinks its running 20 nodes but admits there was an error trying for 24. Doesnt say much about the error - though I have still to look at node-launchpad/src/components/home.rs:401
2024-05-30T22:16:50.194874Z DEBUG node-launchpad/src/app.rs:139: HomeActions(StartNodes)
2024-05-30T22:16:50.194884Z INFO node-launchpad/src/components/home.rs:205: Running maintain node count: 24
2024-05-30T22:16:50.195571Z INFO sn_node_manager/src/cmd/node.rs:588: We are adding+starting 4 nodes + starting these services: ["safenode17", "safenode18", "safenode19", "safenode20"] max_nodes_to_run=24 to_add_count=4
2024-05-30T22:16:50.196317Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.12.4/src/connect.rs:497: starting new connection: https://crates.io/
2024-05-30T22:16:50.196532Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-util-0.1.3/src/client/legacy/connect/dns.rs:122: resolving host="crates.io"
2024-05-30T22:16:50.197541Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-util-0.1.3/src/client/legacy/connect/http.rs:622: connecting to 143.204.194.57:443
2024-05-30T22:16:50.211340Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-util-0.1.3/src/client/legacy/connect/http.rs:625: connected to 143.204.194.57:443
2024-05-30T22:16:50.211412Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/hs.rs:78: No cached session for DnsName("crates.io")
2024-05-30T22:16:50.211517Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/hs.rs:131: Not resuming any session
2024-05-30T22:16:50.225665Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/hs.rs:605: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-05-30T22:16:50.225697Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/tls13.rs:144: Not resuming
2024-05-30T22:16:50.225859Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/tls13.rs:389: TLS1.3 encrypted extensions: [ServerNameAck, Protocols([ProtocolName(687474702f312e31)])]
2024-05-30T22:16:50.225879Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/client/hs.rs:469: ALPN protocol is Some(b"http/1.1")
2024-05-30T22:16:50.826122Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-util-0.1.3/src/client/legacy/pool.rs:397: pooling idle connection for ("https", crates.io)
2024-05-30T22:16:50.860730Z DEBUG /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.22.4/src/common_state.rs:492: Sending warning alert CloseNotify
2024-05-30T22:16:51.018604Z DEBUG sn_node_manager/src/cmd/node.rs:101: Parsing peers...
2024-05-30T22:16:51.018624Z INFO sn_peers_acquisition/src/lib.rs:88: Using peers supplied with the --peer argument(s) or SAFE_PEERS
2024-05-30T22:16:51.646554Z ERROR node-launchpad/src/components/home.rs:401: Error while maintaining 24 running nodes
0: Failed to add one or more services
Suggestion: However, any services that were successfully added will be usable.
2024-05-30T22:16:51.646593Z DEBUG node-launchpad/src/app.rs:139: HomeActions(ServiceManagerOperationCompleted)
2024-05-30T22:16:51.646971Z INFO node-launchpad/src/components/home.rs:82: Loaded node registry. Runnign nodes: 20
Like such?
OK so the problem is NOT that launchpad will only ever display 20 nodes - ticks that off the listâŚ
CPU usage? I am adding another 10 to my 10 nodes and the cpu usage (4 cpu, 4 thread) hits 100% as it starts them all at once. So nodes falling over is more likely under launchpad than using the manager with long interval. [EDIT:] Waiting till cpu settles before adding more
[EDIT:] added one more when cpu was down a tad and it added OK. Will try 4 more when CPU settled more
[EDIT:] CPU down at 50% so added 4 more and have all 25 nodes running (this is on my Odyssey SBC)
Not CPU usage AFAICS - the graph shows me repeatedly pressing ctrl-G half a dozen times - it goes from maybe 5 to 10%
Now this is a Ryzen 3600 with 32Gb RAM, not an old dual Xeon but thats not itâŚ
I have still to clear out / and move a lot of stuff to the backup machine. I can get rid of 100s of gigs of scenery for my flightsim as now I have got fibre installed, I can load the scenery on the fly as I fly
So lets see what if anything changes when I present launchpad with 7-800Gb free rather than 300. Which should still be enough to load 60 nodes anywayâŚ
BTW, my OS is opensuse if that makes a difference
and with 25 nodes its settled down to around the 60% cpu usage, Iâve had 30 on this machine and usage was lower so I expect it to drop more as the new nodes establish themselves.
Also the nodes start with home-network using launchpad. Not sure how it worked out it had to since autonat isnât there yet, or is it? Starlink doesnât allow uPnP or port forwarding
Anyhow I am going to drop it back to 10 nodes since the nat table get overwhelmed with 30 nodes and takes a long while to clear out even when reducing nodes. Internet becomes sluggish with too many even though plenty of b/w available for 30 nodes
Im at 3-4% with 16 nodes. Even when I was running 80 nodes, it was my ISP router that was giving up, CPU usage simply was not a factor.
I set one of my laptops as exposed host
from my router and I am now starting nodes on that to see if thereâs any difference in shunning etc. compared to upnp
and homenetwork
.
Some notions as I start my nodes in sets of 3, waiting the CPU load to cool down in between.
1 should give you 6 nodes
2 you only need to enter this once, but it doesnât hurt to hit D and check that your discord name is still displayed
3 see above
4 the UX will get tweaked soon enough, for now the focus is on core functionality - someone (@joshuef ?) said so the other day - an associated discord name column is a great idea.
Unless you have deliberately nuked ~/.local/share/safe/launchpad/config/
then the last selected dedicated disk space and username will persist between runs.
appears to be working for me
you need to use the power shell to install safenode manager and then use safe up to install node-manager.
I has the same problem but on Win11, I am parked until the GUI and matching release shows up, then I will start in on it again.