i may upgrade NTracking to handle more nodes but the timings for bothering the metrics of each node are the main issue.
currently NTracking checks all node stats every 20min and with a 1 second sleep bettwen nodes which gives us 16 min of sleep time during a node status check and 4 min for the rest of the script.
if you managed to get to 5k nodes with ANM im pretty impressed but im guessing its some other bottleneck you are hitting due to all the process switching.
can i ask the spec of the machine you ran 5k nodes on ?
Epyc 7R43 (equivalent to Epyc 7763: 64 cores Milan), 1TB Ram, 4x16TB Hdd + 1x2Tb high end NVME SSD (all nodes are currently on the ssd).
I can run 2K nodes smoothly (50% cpu usage) on each of my E3-1535M v5 (quad core skylake mobile cpu), the Epyc 7R43 is easily 16x more powerful so it should theoretically handle easily 30K nodes.
Isn’t the limit per process?
My current limit is 1024/process
I currently have 7633 nodes on one machine in dockers, and I’ve started testing with Shu’s slightly reworked non-docker launch script to see how many nodes it will launch on another machine:
Oh how they almost kicked me out of the house I started fancontrol and chatGTP forgot to tell me that after configuring it I have to start it with sudo systemctl enable --now fancontrol and for the 10 minutes until I figured out what was going on the fans were howling at maximum. I will definitely leave it at 85* just to be quiet…
I have a Odyssey SBC (4 core, 4 thread, intel cpu, 8GB) and running 200 nodes with <50% CPU <5GB RAM
It is deliberately kept at 200 nodes because I am running 4 of them on a 30Mbps max uplink starlink. Should really increase each to 300 nodes since the SBCs can prob handle 400 each maxing out CPU&RAM usage, and the starlink can easily handle 1200 nodes on the uplink. I might later on, but doing some other testing stuff using them
Oh and each has only a 18 watt power brick, but not using that much
And they sit in a 3D printed 10" rack with 2 MikroTik routers and 2 tiny fans at the back circulating air (a little) and on a bookshelf, so warm to the touch, but people temperature, not over 40oC. Each device is only warm to the touch. About 5 to 10 degrees above ambient
I have neglected my home rig for a little while, started poking around this morning and if my tests work out as expected I could run somewhere between 15k and 20k from home with plenty of spare capacity left over.
Holy smokes, nothing like taking a hiatus to fully appreciate how things are actually improving.
Currently running 15K nodes on this setup (Best Safe Node hardware - #1110 by d3su), but it’s sitting at 25% CPU and 35% RAM.
Some VMs have weird behavior, so I’m monitoring and still learning on how to improve it.
