Ha good! You had me browsing 2.5 Gbps switches for a second 
.
But for now I just want to see where what I have gets me. Upgrades can come as bottlenecks are found.
Going to be interesting to see if cloud nodes start getting into trouble for stressing systems with large state tables.
I was looking at mikrotik routers a few days ago and got disappointed with routing speeds and cost. Somehow I missed this one, maybe because I steered away from the bare boards and this was excluded too.
That is a nice unit and I doubt I will ever get to 3 gigs or higher internet.
So I checked if I could just ditch my ISP router altogether, and it works out I can. If I want phone then I can even connect it to the 3rd party router. They use a closed shop VOIP system and all I need to do is open ports to the ISP router if I want the phone, but I donāt LOL
So I have now ordered it, Be here in a few weeks since it comes from the US. I ordered the PoE version with PoE+ on all 8 ports which gives me even more options. Now the house can use that router and Iāll route into that one from my gear through a linux based router setup in a small SBC with pihole, opensense etc. Be nice to isolate my gear from others and other devices.
Thanks for posting that, I was starting to think Iād have to put up with the ISPās router
Hey Josh,
As Qi said, the Timeouts are expected whereby a connection is closed if it is inactive for 30 seconds. Also we manually prune connections at even shorter intervals.
But if youāre running nodes for the PunchNet without home-network specified, your node acts as a relay server, with almost uncapped resource allocation. I.e, alot of home nodes can get connected to you + a lot of connections can be opened between home node <-> you <-> others (though hole punching should kick in to reduce some load?!)
Ahh, that explains it.
I guess as a public good running a few as relay then is the right thing to do but all may be a stretch.
Thanks for the explanation.
Will delve deeper and see if this pans out after time. Now that I know what to look for and expect.
Might be good to look at pocketing some Mikrotik shares if this Autonomi thing kicks off 
Some years ago, I had setup some cameras for a auction place and they wanted blueiris to handle them all. So I had a PC handling them which also was reachable from the net. So I told them that they needed a router to isolate their business computers from the open wifi and camera PC so that no one could come in and use their wifi to hack in. Got a small mikrotik to prevent access. Bit of a learning curve at first since they are quite different to mainframe routers or CISCO in commands or gui.
I am looking forward to wrestle with their different terminology. Maybe they have evolved the routerOS to be more familiar. But still I am finally connecting my old (but never used) 24 giga port managed switch and because of indecision it too is a bit of an effort LOL. I need to bite the bullet and just swap out the old managed switch and put this in with downtime while I do it.
I have liked though the quality of the mikrotik managed switches/routers with their ācheapnessā of price for the same quality/abilities as other brands
Iām thinking of upgrading our home router to this one:
But I donāt find information about the maximum size of state tables anywhere. Is this box familiar to anyone?
Also, I find it a bit disconcerting if there is really a need for high-end router to run more than a couple of nodes. What do others think of this?
This might get to several millions some day. Is it really necessary to contact them all?
Also, am I right in my thinking that when the network is small, it is likely that the many of the inital chunks come from the same nodes, so no need to so many unique connections?
But when the network is very large, it might be the case that none of the initial chunks share same nodes as their close group? So, if the network is about 50% full, then the nodes should carry about 1000 chunks on average. Then the maximum number of connections per node would be about 5000 connections on startup, just to load the initial chunks?
I am not a fan of vendor lock in for routers. Nothing like building a generic PC with multiple NICs using commodity parts, or exotic parts if one desires, and running open source router software on it. Any part off that device is replaceable and upgradeable. Plus, its more fun!
While many vendors make routers, very few allow for a High Availability router setup (2 or more physical hosts) with floating VIPs between the two. Having your own custom router (hardware + software) gives you that flexibility if so desired.
As for switches, I bought them used years and years ago, but they are continuing to work well. I have 2 groups of switches, group A consist of 5 switches (48 port each, + SFP+, + HDMI). Group B consist of 3 switches (48 port each, + SFP+, + HDMI). Each group is interconnected, and a failure of 1 switch allows the PCs to continue to maintain uptime via re-routing via other switches. The two independent groups of switches (5 & 3 sets) are interconnected with 20Gbps.
The switches in addition use HDMI cables to create a 10Gbps backbone (ethernet over HDMI) circuit in between each group, while freeing up the actual SFP+ & Ethernet Ports for the servers (essentially HDMI stacking).
I went this route years ago because it was a very cheap solution and allowed for a decent amount of options.
No a concern to me. The network is not asking us to run silly numbers of nodes
We do this to ourselves.
But if people can not run many, then we will need more people. The rap is that we have so much spare space on our devices, but can we really put it to use?
I think most people should easily run 10, 20 maybe 30. For the average user that is plenty no?
The more you run the greater the responsibility too in my opinion. Especially in early days when the network is small.
A slight barrier is probably good.
Precisely.
Folks decided to stick to 1 or 2GB off storage per node, and make that the building blockā¦ for better or worst, it comes with its own set off pros/cons when running N safenode pids from home etc.
If you exceed the typical normal use case or setup, you are on your own (including support and technical burden). Kind of expectedā¦ if you are the edge case, and not the typical safe node operator.
I am still curious just how aggressively is the discovery process for a public nodes (which offer relay services), does it really need to make connections with everyone else or its only a small subset of nodes it need to discover and keep track off over time. How will this pan out with larger and larger network size, say a testnet going on for a month or two (ever increasing)?
You have some serious home networking there. Mine is a lot more basic and I am just making it a tad more complex by separating the house stuff (TV, guests, cameras) from my working gear.
The mikrotik router Iām getting is replacing the ISP router and the main router for the house.
Then I have the device + 2 network card plugged into the M.2 slot (the other m.2 slot has the drive) giving me 4 network ports at 2.5Gbps. Then install proxmox with opensense, pihole, and maybe another server. One port connects to starlink directly, one port connects to the new router (dual homed in effect) and then I have 2 ports for connecting to my gear.
My gear will have a 24 port netgear giga bit switch (going from a 16port) and then a few unmanaged switches with 2.5Gbps or 10Gbps or both - these are to connect my devices with 2.5Gbps and NASes at 10Gbps together and back to the 24 port (or to router I am building to get to starlink) for internet access and/or other 1 Gbps devices to get access to the NASes
Still not much compared to your setup. Gone are the days where I could work on networks spanning 1000+ PCs/devices.
Sounds enough, if that is the case. 
I think Iāll change the title of this thread to not confuse folk into thinking they need new routers.
@Josh Thank you for starting this thread.
I have been stumbling onto the conclusion that my ISP router was contributing to many of the errors my nodes were generating.
I finally found the evidence in the routerās GUI (it was kind of buried):
I observed this count peek to this high while starting nodes. It appears that the router begins to prune connections to stay below ~29000 and then eventually drops the ip of the machine running the nodes after about 12-24 hours if the load does not decrease.
Without further testing, I conclude that for my router: 100 nodes is too many, 50 nodes are barely ok if they can survive peek network events, and 25 nodes are solid and do not affect other devices. @Shu said it best.
I was planning on replacing my ISP router but now it is my priority. 
Does that explain why the nodes I started without the home-network option donāt earn anything?
100% agree that a new router is not required for casual network participation/usage. Just those running extra nodes will have to look into this as a limiting factor.
This I think would be true if you are truly being limited by your state or NAT table entries at your router alreadyā¦ but if thats true, both groups of safenodes (with and without --home-network flags) would be impacted regardless as your router would randomly decide to drop connections to stay within its upper limit.
For me at least, with a very high NAT entries limit set on my router, and no longer a bottleneck, I actually saw more stability with public nodes (w/o --home-network flag) than with private nodes (w/ --home-network ) on the PunchNet testnet.
I hope the scenario improves over time with future testnets here for private nodesā success rates.
It could very well be folks public nodes not on the cloud but at home are running into these router bottlenecks and causing cascading issues serving as relay nodes for private nodes (TBD) due to random router connections being dropped.
Perhaps, with UPnP and AutoNAT fully baked in at some point, incoming connections will simply work for most part for most folks, and therefore more nodes will be basically treated as public nodes, as oppose to private, and with more public nodes out there, they will each have to do less relaying related work? (TBD).
However, keeping a public or private node healthy is up to the node operator especially with respect to their router limitations.
Are the other ones still earning? No nano at my side the last 3 days for ~300 cloud nodes where connection shouldnāt be an issue at all
I havenāt earned anything recently, but faucet was also down for daysā¦ so maybe uploads/earnings have come to a stand still as newer data isnāt being propagated or uploaded.