Nat detection still has some unnecessary issues

[neo]

@roland @bzee

This all started when I was wanting to find an easy way to see if a node setup using direct connection is correct, be it via port-forwarding in the router or say a VPS with public IP address for the VPS. (EG @Dimitar is doing this with PC directly connected to internet and no router)

The issues that can stop connection can include

• port-forwarding in router not configured right (ie tcp instead of udp, or incorrect port number or incorrect LAN ip address)
• firewall in device incorrectly configured or not at all and the required ports are not open for udp
• node manager not being told the ports to use for listening, or incorrect ports specified. (--node-port)

When asking around Joshuef showed the function where a peer is asked for a quote (requiring the multiaddr) and then @roland told me of the nat-detection command in safenode-manager and a list of servers to choose from.

/ip4/138.68.131.45/tcp/17709
/ip4/138.68.131.45/tcp/17710
/ip4/138.68.133.45/tcp/17700
/ip4/138.68.133.45/tcp/17701
/ip4/138.68.133.45/tcp/17702
/ip4/138.68.133.45/tcp/17703

Some of the issues I came across using safenode-manager and the nat-detection command are that

• upnp was not detected as available on a computer that successfully will use the --upnp option in safenode-manager
• direct connection was not detected on a VPS w/firewall that uses it successfully.
• starlink (forwarding and upnp not available) it detected correctly

In all 3 cases the safenode-manager nat-detection command said it was private. This means that the launcher would set home-network for all 3 cases.

This lead me to realise at least one thing

• The port number being used by the nat-detection command in the manager is using a random port to test to the servers and thus will never be able to be successful for a setup that has portforwarding and/or the firewall set correctly to only allow the ports needed for nodes. Also correctly setting the protocol
• that there is either a problem with upnp or the way it tries to upnp in my router that is different to the way nodes use it.

Now today @chriso relayed a message and that I can set the port number in the nat-detection program.

So I used the nat-detection program that the safenode-manager downloaded and used the port number on my VPS (opened in f/w for udp traffic). Tried first time and got error for upnp not available which is expected since there is no router with VPS that has public IP, then ran this

nat-detection -n -v --port 40000 /ip4/138.68.131.45/tcp/17708

This gave no output and just returned to the prompt. So I ran it again echoing the return value from the command. It gave a value of 12. But without knowing what that signifies I don’t know what it found

But thinking further I realised this would be a sure failure as well since the servers @roland gave me (see above) would not work in my correctly setup firewall. The servers are for tcp, and I only opened udp

Thus using those servers will result in failure for anyone with correctly setup firewalls and/or routers trying for direct connections and using the launcher.

Also for being able to check to see if the setup is all correct it will say that homenetwork is the only option.

Thoughts opinions. This is a summary of what I’ve seen but hopefully gives enough information to work on.

[storage_guy]

Well done! That seems to tie in with what I posted here:-

Assemble at the Start Line. The Beta is About to Begin

A whole day without a message in this topic so things must be running well! Or we’ve had to at least pretend to do our day jobs for a bit! I’d like to run my nodes at home in the --node-port mode and not using --home-network so that rather than adding to the load of relays I’m being a relay to others. I’m doing that and it seems to work: I’m getting PUTs, GETs and occasional nanos. I think the port forwarding is working fine on the nodes at home or surely I wouldn’t be getting these and woul…

that even using the --port option seems to result in my nodes at home being set to use a relay and not being a relay themselves because if they fail the test you mention above they will be thought of as using the --home-network option.

[neo]

safenode-manager does not use nat-detection, it just has the option to run it if asked. Safenode-manager had to download the nat-detection program in order to execute the detection command. Yet I had been using safenode-manager to run nodes with --node-port previously

[bzee]

@neo

that there is either a problem with upnp or the way it tries to upnp in my router that is different to the way nodes use it.

nat-detection uses the same UPnP implementation from libp2p, so that would be an interesting thing to figure out.

@neo

This gave no output and just returned to the prompt.

Try running it with -vvvv instead of -v.

@neo

The servers are for tcp, and I only opened udp

This is actually a user experience thing I hadn’t realized at all. I assumed people would forward both, but I realize you can open only for one protocol. The annoying thing is that the AutoNAT libp2p implementation is broken on UDP, that’s why we can only do it for TCP.

@neo

Thus using those servers will result in failure for anyone with correctly setup firewalls and/or routers trying for direct connections and using the launcher.

If you open ports, those ports will work, but your router will still be NAT, so technically it does detect what it needs to detect. But I know this is a stupid way to look at it, because when you forward ports, you care about those specific ports.

It gets messy with port ranges. We would need to do proper port detection for the whole range to make sure all those ports are readable e.g. By that point I would just think that we should allow the launcher/manager to skip nat-detection. At least, that’s the ‘expert use case’, and we didn’t write the tool for those people. If someone is forwarding ports, then he should know what he’s doing.

[neo]

@bzee

If you open ports, those ports will work, but your router will still be NAT, so technically it does detect what it needs to detect

I disagree that even technically it is not working. It is not crashing yes but not working at detecting the connection you should have.

And as a tool to detect if your nodes can be contacted it does not work.

Anyone doing port-forwarding should already know best security is to only forward ports needed and only for the protocol being used

The same for opening ports in your firewall.

@bzee

It gets messy with port ranges. We would need to do proper port detection for the whole range to make sure all those ports are readable e.g. By that point I would just think that we should allow the launcher/manager to skip nat-detection

Think about it for a little longer, you use one of the ports specified by the user, or the random port the safenode-manager is choosing. Yes this might require the safenode-manager to select the port to use if the supplied node-port is 0, instead of the node. Maybe leave the logic in the node if people just run the node s/w directly.

@bzee

By that point I would just think that we should allow the launcher/manager to skip nat-detection. At least, that’s the ‘expert use case’, and we didn’t write the tool for those people. If someone is forwarding ports, then he should know what he’s doing.

The whole reason this came about was the suggestion by Roland to use it to do the testing of connectivity, or at least suggested it based on my requirements which was the same as wanting to be able to check someone’s connectivity, that a client could connect to the node, or at least one of the nodes. That is a major support issue we have to answer.

Ideally we need to consider that some of these people doing “expert setups” are piecing together information from what others have said and attempting direct connections without the expertise needed to know if it works. And even those with expertise may question the setup they did if they haven’t got earnings yet (or even a quote requested from the nodes). This would be a valuable tool and can even be run to double check the users setup as they add or start their nodes in direct connections.

This would save a ton of questions.

But only being able to do tcp, means you cannot test upnp for all cases since its possible to have unpn for one of tcp or udp too. Any chance this can be corrected in the future?

[bzee]

@neo

This gave no output and just returned to the prompt. So I ran it again echoing the return value from the command. It gave a value of 12. But without knowing what that signifies I don’t know what it found

12 signifies it is private.

$ nat-detection --help
A tool to detect NAT status of the machine. It can be run in server mode or client mode.
The program returns with the following exit codes based on the network status:
- 10: Public NAT
- 11: Public under UPnP
- 12: Private or Unknown NAT

Did you forward TCP too? I would be curious what the output is with -vvvv.

@neo

And as a tool to detect if your nodes can be contacted it does not work.

I disagree with this partially. Yes, TCP vs UDP makes it technically broken, because it only tells you the TCP status while the node is using UDP. But it does detect reachability for a port very reliably ohterwise.

@neo

The whole reason this came about was the suggestion by Roland to use it to do the testing of connectivity, or at least suggested it based on my requirements which was the same as wanting to be able to check someone’s connectivity, that a client could connect to the node, or at least one of the nodes.

It might be an idea for us to add a probe command to the autonomi command to check if you can reach a specific node. For now, I think nat-detection with --port will already be a huge help to most people trying to predict their connectivity status when running a node.

@neo

But only being able to do tcp, means you cannot test upnp for all cases since its possible to have unpn for one of tcp or udp too. Any chance this can be corrected in the future?

Specifically for the UPnP implementation in libp2p, I consider the chances low . But, work is being done on AutoNAT v2 that fixes the UDP issue. (feat(autonatv2): Implement autonat v2 by umgefahren · Pull Request #1 · umgefahren/rust-libp2p · GitHub)

We’ve put our hopes on AutoNAT v2. Our current nat-detection is based on AutoNAT v1 and these issues you describe I also talked about here: NAT Traversal ETA? - #5 by bzee

I feel like nat-detection does what it can. The reason we made it as a separate tool is because we can’t run it as part of the node because we need to pick either TCP or QUIC for the node or it’s going to mix up all these protocols in the network. And we can’t use AutoNAT with QUIC, so that’s how we ended up with the current situation.

[peca]

@bzee

It might be an idea for us to add a probe command to the autonomi command to check if you can reach a specific node.

That would be very useful.

[neo]

@bzee
The tool I use from a different internet connection to see if my nodes are contactable is

nc -u -v -z 1.2.3.4 a-b

where 1.2.3.4 is the ip address and a-b is the port range. Will report the contactable ports in that range

nc is a linux network program (on opensuse at least)

[bzee]

Interesting, how does nc work with UDP? If I use it on a non-existing IP it reports back it succeeded:

$ ping 192.168.1.232
PING 192.168.1.232 (192.168.1.232): 56 data bytes
ping: sendto: No route to host
ping: sendto: Host is down
Request timeout for icmp_seq 0
$ nc -u -v -z 192.168.1.232 40000-40005
Connection to 192.168.1.232 port 40000 [udp/safetynetp] succeeded!
Connection to 192.168.1.232 port 40001 [udp/*] succeeded!
Connection to 192.168.1.232 port 40002 [udp/*] succeeded!
Connection to 192.168.1.232 port 40003 [udp/*] succeeded!
Connection to 192.168.1.232 port 40004 [udp/*] succeeded!
Connection to 192.168.1.232 port 40005 [udp/*] succeeded!

[neo]

Being a local/private IP address I don’t know. Your router could be doing something too

But when used on public IP it only showed the active open ports on my other PC on another ISP connection

I just tried it on a random IP address out there and it succeeds as well. So I dunno, guess its not that successful. So a Autonomi test tool is needed where the node responds with something that proves its can respond

[neo]

@bzee

The very first thing on this journey of trying to help people know if their node is working right was to ask about getting a quote from a specific peer.

Getting a quote from a specified peer means that the node is operating as expected and is contactable. IE the person has a working node to the best of our knowledge

That would be a great way to know if a node is good and contactable since it also makes the node do functional work.

How hard would that be to modify the client code to do this. (I have not gotten into rust)