I've noticed an increase in problems reading the network-contacts file

[neo]

As per the title

Background: problems reading the contacts file were rare in the past and just having requests to read it blocked for a period.

Lately this has been increasing and there were some in the discord unable to start any nodes due to this issue and they were unlucky to hit the problem each time they tried. I have been noticing it more lately with at least one to 5 nodes not starting with issues reading the contacts file. Come back later and they will start. This last testnet now closed down, it was start 3-8 nodes then all the rest would not be able to read the file off the server. Have to wait 10 minutes or more and try again only to have the first 3 to 8 start and the rest fail.

Suggestion:
Is it possible to have nodes when they successfully read the contacts file then write out locally the file to disk.

Then that local file can be used as a backup OR primary source of contacts when starting. If as a backup then when the attempt to read the file from the internet it then reads the local copy. OR use it first and if not yet there go to the internet for it and write the file. This means someone starting 20 or 300 or … nodes will only need to contact the server once to get the file.

The file name would have to include the current version so that a later test (or live) network will not look for that file but its own version.

There maybe the need to have a option to force the read from the internet and rewrite the file.

If the file is used as the primary source then this would greatly reduce the load on the server supplying the file and reduce the chance of people being blocked from reading the file due to some sort of ddos protection or because of the load on the server.

[drirmbda]

I’m caching the contacts file and using the entries to specify peers on the safenode command line, using my own script, as I have seen this annoying per IP rate limiting behavior in the past too.

In addition my launching script checks if nodes succeeded in obtaining contacts because unfortunately nodes just keep running uselessly after failing to get contacts.

I consider this node behavior a bug.

[neo]

Yea, for general use using the peers env is not best since people don’t understand and then when they upgrade they forget they did that and wonder why they cannot connect.

But I agree that is a good work around.

That is one reason I am suggesting that nodes write the contacts file locally on their disk where other nodes starting can look for it. Makes the setting of env unnecessary

[drirmbda]

Another work-around is bootstrapping off your own healthy nodes that already are part of the right network. What I learned from that was to use more than one to increase bootstrap success rate.

In the past the rate limit appeared to be something like one request per 5 minutes. If this limiting is on purpose, more transparency from autonomi about this would have saved a lot of time.