Unable to see other nodes on the same local network when uploading a file

Situation:

Running nodes on the same computer and or same network as the client

When uploading a file that happens to access one of my nodes that node gets rejected with a erorr other - connection timeout.

This will be due to hairpin routing not happening in the router where the router is unable to redirect a packet send to the public IP address of my connection back into the local network. The iSP rejects packets from an IP address being sent back to the IP address.

Consequences:

Any large node “farm” behind the one router/public IP will have degraded network connectivity when a client tried to connect to one of the nodes within that “farm”. I also suspect that nodes within the “farm” will have trouble connecting to nodes within that “farm” as well since that uses the same routing technique.

Another consequence is that if 2 nodes are close to each other within that “farm” then they will never see each other as a close node and their routing table will have to reach out that little bit further.

Fixes - are they needed

Not sure its needed in a large network since the client upload still has other nodes to choose for uploading.

Also if two nodes are close to each other within the same “farm” then all that means is they ignore each other and still have have many other nodes to connect to.

Only time it would be an issue as far as I can tell is when either the network is small which relates to the other option that the “farm” is a significant size of the network causing significant reduction in connectivity of the small network.

why mention it here?

There maybe other consequences that I have not seen. Also it might prompt a dev to consider the issue even for a moment

And might help explain why with the beta we had the lottery winners since we had some “farms” on the order of many thousands of nodes in a 40K node network (less at times). With that very significant proportion of nodes having connection issues to each other it’ll throw off the “close” node situation. yes I realise other reasons were involved in causing it, just saying this may have made those other conditions more effective in causing lottery winners.

7 Likes

Yes, we can turn on mdns etc. to “fix” that, but likely other consequences with that too.

It’s an interesting one, do we want to make farms easier to have or not?

Without saying either way, it’s the elephant in the room I think. Many folk are running farms and it’s amazing and great to see, but overall don’t we want that limited and home users to be the main supply?

Home users are underserved technically right now with lack of ipv6 or NAT traversal. So saying home users only is not really feasible right now. So it seems we kinda need at least some farms, but does that lead to centralisation?

7 Likes

Yea, my conclusion is that its an interesting situation and should be in a file/database of things to know. Know in the sense of knowing it exists in case something in future comes up that this helps explain.

I agree with you that while “farms” have their place in having a variety of node setups and many of us will be running 50 or more nodes. (I am on the low end due to upload bandwidth limitations) BTW I saw port forwarding working on starlink using a mikrotik router replacing the gen1 router.

This last update has been a real positive for home nodes with all sorts of home setups now working, and the network is almost double the size. Sounds like home nodes are now over 1/2 the network, maybe 3/4 at least. That is good news.

5 Likes

Also ipv6 is fine there. Man I wish it were rolled out and NAT was dead

100% it’s looking great

4 Likes

Do we know, or have any idea, when IPv6 will be used by Autonomi?

3 Likes

It’s not easy, we can enable it quickly and simply. Issue is what to do when some nodes are only ipv6 and some only ipv4. So there is an issue to get over, bit like NAT traversal.

I am mulling this and dreaming of ways to have this. Not one seemingly simple mechanism works, it’s deep stuff and needs a lot of thought. It’s where the community is great but hard a many folk will knee jerk nd ask questions and then there is no time to think and so on. I am looking though for ways around the different topology questions

7 Likes

https://www.google.com/intl/en/ipv6/statistics.html

5 Likes

I think you ought to concentrate on getting things right for home users for now. That is what we really want.

I think there will kind of be ‘farms’. But not how we think about them for Bitcoin and Ethereum. I don’t see farms working because of network bandwidth being the limiting factor. Unless something changes drastically with networking technology and the circuits supplied to DCs increase in available bandwidth by a couple of orders of magnitude that will always be the case.

But I can imagine companies who are using Autonomi to store their data wanting to run some servers with Autonomi nodes on them to defray their costs and usefully soak up spare bandwidth. But only the amount they can run without overloading their bandwidth in and out or sessions on routers because we know those are the limiting factors.

I don’t really see that as being a centralisation problem though even if in total they are something like 10% or 25% even of the network. They are just large users and not trying to constantly outdo the other ‘farmers’ like Bitcoin and Ethereum miners do or even trying to control 51% of the network.

So when these large users come along maybe something should be done to make large sites possible without causing problems either for their nodes or the network in general because of their nodes not being able to see each other.

7 Likes

For advanced users, you can enable NAT Reflection or Pure NAT or equivalent on one’s routers, and that fixes this issue. I did this in the past testnets (months ago) on home nodes to reduce error rates, especially when network size was much smaller, and my nodes needed to communicate with my other nodes on say 2 different physical hosts but with same WAN IP but different ports etc.

3 Likes

Correct. And I had just come back to do it and read the forum first

Although people do not generally do this, nor even know about.

And some do not have access to the router.

Reason I presented it to be something in the “things to know” list

3 Likes