Shunned

aatonnomicc · April 13, 2024, 9:30am

I started 50 nodes with a 4 min interval last night and am now seeing much shunning going on

trying to see what I can find in the logs for you but grepping for SHUNNED is not producing any hits
is there anything else I should look for ?

ubuntu@s00:~$ grep "SHUNNED" /var/log/safenode/*/*
ubuntu@s00:~$

happybeing · April 13, 2024, 9:47am

Sorry, i mislead you. Please search for “consider us as BAD”.

Brain malfunction on my part

aatonnomicc · April 13, 2024, 9:54am

no worries usualy i have to mislead myself
that worked here is the out put from the logs hopefully there is something useful there for you

ubuntu@s00:~$ grep "consider us as BAD" /var/log/safenode/*/*

/var/log/safenode/safenode11/safenode.log:[2024-04-13T03:37:31.485033Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWM9XnmSRX9gXGewXS                                                  WWGn7aM7tgMJNZGF6G24EvSK1Bxr - 5b1e58bd0c4f60222ec81ae93ceb83dc9167ebfb83ba02feb64959c21f434388) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode13/safenode.log.20240413T050147:[2024-04-12T20:55:47.250759Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  BD3C8Xy7UABypJcbHWaVKvmG8byLrZ7Wi3hF8B4Bm3K2 - 62a526594273c19cc42630926e9c7159c03c40293b468a1d69724b1a1b0442bb) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode13/safenode.log.20240413T050147:[2024-04-12T20:55:47.250921Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  M9sBN5Y2kGt19s1REzhiSQxaAHEx8jGv3zfBy5jzZ6vE - 635c30874dde4e75b5dd4c76c46e534358e60edf7f18ed26edddcf109acf5b51) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode14/safenode.log:[2024-04-12T20:59:54.441052Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWL2HjMyB7y18KRdbw                                                  fhTy2THCeCwWevwxPJTXaFNX36L6 - 6b72cdf827201602079d94c32af0952075585ce4eb59d95df17191a41af9d32b) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode14/safenode.log:[2024-04-12T20:59:54.441094Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWF97A6uTvdideCxkC                                                  GL6vA3znfqx73ig5ZMfaoX7orz8q - 6a961d0bcda7a0299705c2e1f0c13d0ad6a98c466466096956cb97cc563e85b8) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode14/safenode.log:[2024-04-12T20:59:55.401185Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWCv5ZGVLWAANCJPqt                                                  FkjoF3s41uDeesLtnhREsE686XJ5 - 6b61ad671e0228e561227ac9892958e1d10fb73b9b610eb3f4d16540d2791139) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode14/safenode.log:[2024-04-12T20:59:55.401378Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWHwCeZ1kSha8yh7Lp                                                  RXXiRAfCvZ9xVUqtc8Ek7gxPdHxW - 6a737648e0b21fd251f6b16fa57bbd349e0e735f27de860dbe6f9de8f1366cdf) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode15/safenode.log:[2024-04-12T21:04:14.624835Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWDvh49ofMheM3n3fn                                                  tXiNDQwZ38tzWBL645x1YEsW8KEt - b43dc9e0dd629d21a1899bce675ecd559276c39a15a8b85ed4aca5c05e4bbe82) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode15/safenode.log:[2024-04-13T03:38:23.759636Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWMVz49NHos5KPbqB5                                                  up8MvWMM5yadoG7shgwnLoKkLdfr - b42a4dd212d4ff07846baf8505f8af4c6dbb67f5a44dc01102df2fbafb0ed413) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode19/safenode.log:[2024-04-12T21:20:17.015657Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWAMPHiLguwN2avQCw                                                  mJA8NppQppQpgACCC4jUtmtK9gCX - 604e808f595da4da8f34c1a019e0969d6ccf038a827d0ba339be5aacf4da6804) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode19/safenode.log:[2024-04-12T21:20:17.221229Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPL7RPx7vnVkVwMB6                                                  vQ13RsaDWpsXpyXf99YEb4EvWSvG - 6003bf2bd4d19d0c69090baadbbb20ce45fb0b545c93c56ff0efd6d646e330c5) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode19/safenode.log:[2024-04-12T21:20:19.352038Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA58bgZMhdbV8yu9c                                                  9Rdekb5fMN59mizGnqoSQEnouSLp - 6165c13532ec13a6cd82a7ddf251611294f7e1c1015cf4cd6333560d0415ba17) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode19/safenode.log:[2024-04-12T21:20:19.592837Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPaS56rMALBVPR1rc                                                  nvQuXhDcnWZ4Dm1egpKyZnuPZCpm - 60d67252fcd25cdd2dd586c2a545c0cca2140b52e37f992b8e99265cd85d5dfc) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode19/safenode.log:[2024-04-12T23:47:48.580716Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLjdFoYb4fbbRLdWC                                                  sF7dSKcZEvG1fTZj2UXqZNzUmoK7 - 606191b2b1cd0a689353281b65bc7d531c7c4e8579ffa8299c0688150bab49ce) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode1/safenode.log:[2024-04-12T20:29:19.913127Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWG6ippiKLpGr5Vsvu3                                                  tv3sstewPU55YxTE9nR85uJSug7 - d285e5ff6f2b18087ab733a47d6172035033b1fcde2e09ec8dd912c2b588c912) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode1/safenode.log:[2024-04-12T21:10:29.601217Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWRyXraTU5x5vBQxufU                                                  Ybdxu7XMz8Tv9LRtGZXhgW6jVyV - d208660dd8eb133c2d018de86ef4b72dbbc8253448a4fd54ad8fab4def2b5d51) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode1/safenode.log:[2024-04-12T23:54:10.730379Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWF5idv7UoDNu6LbBzn                                                  aXyRGLXD8zDCHyQnNEwAKhi1Vhd - d1bbe5d06aadb9e8189c717ae0e983ea321a9591aa406efeffd4b1437655ff98) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:24:56.055630Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNadS5U5QYthynT7R                                                  864zRbL23BN8DkjwCHsbyzevN8fZ - 5dc524c542ad6dedc0a9e5a002194d3d3d999edeb3767326e95546f93ad00416) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:36:11.071330Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWGGDCPYp8bss8eftw                                                  8uGwqX9jD1zMtVCCqzYnXoXwh45i - 5d731c80f05f32317961977c07fd8b78d1b605402d5f8ab3e2930ee39f204f1b) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:40:41.036566Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWRchxEW7aSH7MgHwZ                                                  KiJQFAdy9nv2vsfg6QfmYRJqhJXM - 5dc7b1725518ead0dcd2eab211ac2bd33e5c86d7dc2956cdadb4cc8dbdd56d16) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:44:44.058296Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNef5zKGf8D2idnV4                                                  eQWoQ3XNRgU67o2YYN6B7oq2WRYX - 5d995ad4b83bd5b4aeecbedfd34a0936e164834839683d691f2e06f748882469) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:45:38.112037Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWQUet2ssEH2h6qKrS                                                  BEQmk3sP9ZRBUpv2yqJTFCLr2LRT - 5dfdeb5d30349f20c88c0caf642596b6d664efb96fad900ccd402f41270af337) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode20/safenode.log:[2024-04-12T23:50:35.079946Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLm9MwSfe4QJyGVb7                                                  CSLDVXQJDYggK3gw4B9GGh61jJ6H - 5d9d8c60fd8d4da987f8da7510b7a1b632327d32ed3845f6a7a8488bf0a8924d) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode21/safenode.log.20240413T063301:[2024-04-12T22:58:48.326744Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  RpmFhp8eRVQGFGAUG3pDzVRVsysAQKvjHCPfwapQU4wv - ff59c220dc96b72fc16a4abae20b92dd5988a67357888a05ed0893d87e93cc11) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode21/safenode.log.20240413T063301:[2024-04-13T03:37:43.245768Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  JnAVftX1tpCJYnG9xmSDBvxnZpNwMwM5TWwdYSaza2Ep - ff98879098894e9aef2bdbdec4a7223f3b156b057d69896bf975c7295c064058) consider us as BAD, due to "Replicati                                                  onFailure".
/var/log/safenode/safenode21/safenode.log.20240413T063301:[2024-04-13T03:38:47.534713Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  SN1EoEKbzMzwh551T1hQ9oRbGgKYEfCrapL8Ri7aMBx5 - ff9624967fd29e2ee8851f0fa72f17bad5372e938b4d5dc37590026159de1fdb) consider us as BAD, due to "Replicati                                                  onFailure".
/var/log/safenode/safenode21/safenode.log.20240413T063301:[2024-04-13T03:43:14.251547Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  E8C5KGkpyGg4SWqj5hnK3FCMvdaTmLqosgUbBpisCEcT - fec9395f49530a7a896b7d20dfeabac805f15d5906cd599c4d5bc3dfe1060366) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode23/safenode.log:[2024-04-12T21:36:36.878477Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWJfyzjDkLwReyprMX                                                  87kfMZ1HuFkCniQGZ1T7iFnH68rc - 7f346c389c6c39131bac972aaf4ba138700e70743782c94482f2aafba36c6d78) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode23/safenode.log:[2024-04-12T21:36:36.878519Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWBopW9ZJ7h2HxgApo                                                  Nabt3MDQt9T56cjX8DP1ctnwEJhR - 7f5c3251376609e6c86c85103d2d4f2410eeb1d2584897220d34164aa13cc74d) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-12T21:52:54.463909Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWCL7pNF4VjNoJMHki                                                  UvKTWEnNX5qojk5DCL7btZMG13Nb - 617d9fb49d55172f2c4ce9fcacdfa039dda1063706647e2d4f9c161588029e11) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-12T21:52:54.721049Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA58bgZMhdbV8yu9c                                                  9Rdekb5fMN59mizGnqoSQEnouSLp - 6165c13532ec13a6cd82a7ddf251611294f7e1c1015cf4cd6333560d0415ba17) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-12T21:52:55.015259Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWC5tEr7VEtVkL3AES                                                  ASW5HWAeDKaYgxXiFf3aBbAchNKC - 61529f7d75d441cea60f9a9535181b6f55e681d39ab517a24e765ffc1f7279bb) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-12T21:52:56.110959Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPaS56rMALBVPR1rc                                                  nvQuXhDcnWZ4Dm1egpKyZnuPZCpm - 60d67252fcd25cdd2dd586c2a545c0cca2140b52e37f992b8e99265cd85d5dfc) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-12T21:52:57.391970Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNXLfSVqUf6dcQjQz                                                  oc1DzCUWu4kbwm23ra2G2hGM2eQv - 61676d77211864282d86a11268dc72539bb34d88cf5278d5a61387891aed2403) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode27/safenode.log:[2024-04-13T09:44:55.554983Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWAMPHiLguwN2avQCw                                                  mJA8NppQppQpgACCC4jUtmtK9gCX - 604e808f595da4da8f34c1a019e0969d6ccf038a827d0ba339be5aacf4da6804) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode29/safenode.log:[2024-04-12T23:28:25.587878Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWAE7xLJgDv4wXzuGa                                                  45xE3SsYoWoGhcPCLih7x6qABfUQ - bf392dd09dcd58e6a0535bfb7a4efe13322463a79fe8454ee83af5f25f3cf1a1) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode2/safenode.log:[2024-04-13T09:38:20.505144Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWQtm5tQRrizTovrifL                                                  Jm59HqF8uF1CcjKGjq7EbJzmFvK - 4869d63d9735f277f0ffabf88d9d16fb85342785337fad28b5cd3fa11214796f) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode30/safenode.log:[2024-04-12T22:05:17.051858Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWBoZHv8q41xkJhcHx                                                  jq9pBmMYNbdPA9AY4y2m4zmtQGQB - 607cc1c1c3594fed95687e1e8ec003865993392f6e7a988fdaa1f1c39c183601) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-12T22:05:17.051896Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPL7RPx7vnVkVwMB6                                                  vQ13RsaDWpsXpyXf99YEb4EvWSvG - 6003bf2bd4d19d0c69090baadbbb20ce45fb0b545c93c56ff0efd6d646e330c5) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-12T22:05:17.051923Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLGi3D6viR9ErbrbM                                                  CNamECQ9Gab56nqRcqBD3xRerYhb - 602d27f68e5806b2fbf7c270e9e2ae88a0ee5b4ead33c63c70eb5136ae6fa64f) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-12T22:05:17.417357Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPVTJNdxd6HqQKQf7                                                  qQAt6tMzQ7ZavFoPqNCYu9GbQ8me - 601825414e3a4c5d18a260a5bb87cb9edbddcf9f0de2586d3af1f76f47fd2b04) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-12T22:05:17.451113Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWAMPHiLguwN2avQCw                                                  mJA8NppQppQpgACCC4jUtmtK9gCX - 604e808f595da4da8f34c1a019e0969d6ccf038a827d0ba339be5aacf4da6804) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-13T09:36:37.331933Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWCL7pNF4VjNoJMHki                                                  UvKTWEnNX5qojk5DCL7btZMG13Nb - 617d9fb49d55172f2c4ce9fcacdfa039dda1063706647e2d4f9c161588029e11) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-13T09:40:37.971147Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNXLfSVqUf6dcQjQz                                                  oc1DzCUWu4kbwm23ra2G2hGM2eQv - 61676d77211864282d86a11268dc72539bb34d88cf5278d5a61387891aed2403) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode30/safenode.log:[2024-04-13T09:43:48.850284Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA58bgZMhdbV8yu9c                                                  9Rdekb5fMN59mizGnqoSQEnouSLp - 6165c13532ec13a6cd82a7ddf251611294f7e1c1015cf4cd6333560d0415ba17) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode31/safenode.log:[2024-04-12T23:12:00.926699Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWC7oxN2R6R2axu9Mg                                                  EH6i7EPTLdxEZFwr7nGzt5v2kA3f - 6f2895f046dc8afb41b370eb068c48091847ea300ad0c46f0e9a01d236ec6f78) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode31/safenode.log:[2024-04-13T09:36:55.642953Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWDZMv2Gtbacz75pMt                                                  rV64CeWrGxaYnF1DkvmyG4EFCBAR - 6e2d11e99123d8294e236df842f45ae78d466f838c9a785f99a7d3e5ca78b047) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode31/safenode.log:[2024-04-13T09:37:23.137173Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWJuN5FyG4aaNEbcx6                                                  UVavStYSiQDZfSWKCbz5JL7T7X7d - 6eca3b1c1285b15329ebe84e33dd6b906503ce36ae150d462eed97cc4f106d9d) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode31/safenode.log:[2024-04-13T09:37:27.965951Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWM2xknyitDJh11sTG                                                  M1CZj9Fg3UJko9AgVfMiWLDH8H3X - 6ea636701707b763df8542e1ae93715e698f5011c68f7ddb05250258a08ec404) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode31/safenode.log:[2024-04-13T09:37:41.416649Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWC1XT6awuBsZYrZtg                                                  466ZpnTsAHACNTwb7t9s7Qnp3XY1 - 6ea525093fa1e7285fe6e2508840fe7fabeb90703909f2f024fc7527ea6dcd0e) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode33/safenode.log:[2024-04-12T22:17:25.822127Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWKK3t2BNHtNtN3AXR                                                  fiTmUJbPrx19B6tf3mBGKpShbSFt - 62646377971358304a6bc4e1997f8998f1e56a6df0509361989c4058510aa5d0) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode33/safenode.log:[2024-04-12T22:17:26.090629Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWFqCT1VnkvzLBpnne                                                  hZBhiR4nRoizvdAnbVhEwBaSFiW3 - 629c63604bde6fd8a97a3e93d69e51b6639298664ef504b528d60a87960d6474) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode33/safenode.log:[2024-04-12T22:17:36.152265Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWGovzUfDXyHASmz33                                                  WGadrF7TB4gH3id5zV6EGDr1CdU7 - 622838b5a64517519822dd7bc9635b7d783b736809c5b12f1306d62bd87431c1) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode33/safenode.log:[2024-04-12T22:18:19.423745Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWPCE1ZHaxF1k4pFnG                                                  8HRLKR1hY5zFCNCzZB3qNY6YYVYf - 629af0d9d226345dd6ae4c5d19acd74e2569e55da30993c6972e67a199f1ec2b) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode33/safenode.log:[2024-04-12T22:18:28.066580Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWM2yKekaMwYAvHQNc                                                  45kP5pH58MP4JCGXwzSz9zbexLzC - 622c79a258dc325fb451a6b9e33a491791810b434c6fc69791470ab0c616b698) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode35/safenode.log:[2024-04-12T23:53:25.078261Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLG4vaLMHjH5TbNSw                                                  EcFMmtNh2cXrsY72kz4bBsnY1ZSA - 82d49d6cb2899d599210e21a1d2c0cc60120ed52fe360f9aa869eb2ed02bd033) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode35/safenode.log:[2024-04-13T00:03:08.729037Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWSp7fE1wK19JY8uYn                                                  LDJssksQixbPA2tgw6xzGL2SXSVo - 832e049abb803059cf30ab6bb0030f2d68df989250d2cb3e95a9216f1fff758c) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode35/safenode.log.20240412T225336:[2024-04-12T22:25:54.045242Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  NffUoLTSMmzvPfroJxrZa4wDTdjcuwYD6Lz9om1sCawx - 8252ada69f4a77140adb3594890f0ee83f65533ccb4fe131e43b38b504bdc9f2) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode35/safenode.log.20240412T233523:[2024-04-12T22:55:37.335429Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  Q7S9qM2gLoaxYynvkPXWvVZ9s9D34YMVTUFNrRNtKwpv - 83cea6749728117ebd7e2a675c604e9bf68ee154656bd853d62b3a8857797ca9) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode37/safenode.log:[2024-04-13T03:39:48.389467Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWK3C9V4D2w4gBcuZz                                                  w1SxNKwipb7Rad32hkzUWyahiUdp - 38cf12c058c66a4fd336bb1951bb14d1bc7ce4af9dd430b0c9ded11120e5ed29) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode38/safenode.log:[2024-04-12T23:32:29.884569Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWEETrF8BkKzdyUd6k                                                  CAo8QBR9sTbTE5ZTW14LjBF4qrKS - 50b90be04063d63ed2830c173e42bcbe5d359131e8a1df29ebbbcc245b61647e) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode3/safenode.log:[2024-04-12T20:29:45.274846Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWMw1Vv62CsCMzwnyno                                                  Z87gjdkTdERzXF5gVBu5WgPoTVr - 88535497521f535b13dea199a55de34be8ac106e205756f79805f312242f1169) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode40/safenode.log:[2024-04-12T22:46:04.459712Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWQg3caciqEtTRWZmJ                                                  cUJXWF1Phw5MrCvVeFkUsaRmTdpC - 7b5bc0449d473cfef52e224c51af5cacbf17c9bf4f947776bc7651410f170dc1) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode40/safenode.log:[2024-04-13T03:36:12.274066Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWS1FL6Y39HGQ6g9ZY                                                  6vyNJKXfoWWoZHsFyotDgHU2QQH4 - 7b1e157c72ab2407ba522e14f62198d7f205d00941865983ccd4aa482f3e1e4b) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode40/safenode.log:[2024-04-13T03:38:11.799942Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWG5QZFonnDrTN7A8a                                                  Lukbt7tHDJRjXFEy1Mi7sYqSYnsx - 7a21e5c908d56923bffaa1901a4fc8068a939d420baf2f35f0a042c204a3ada9) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode42/safenode.log:[2024-04-12T22:53:57.601880Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWMjFeZ8MRevsU3DYk                                                  E7MboLMUAiDeRstQuj38jCNBAwCc - a1bf42287e95271d17b7c284f4b5d373bbe964067d5f6a19bf4ffa87360a2950) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode42/safenode.log:[2024-04-12T22:54:01.098006Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWGQ6kLJjt5s9BQVR7                                                  uJbTCUD2s95uzbnDXMvTCy2u2c4e - a1828c729aabe0db134f3d3f9ef9b6ba4b74536873045ff7cb7a0263cd657715) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode42/safenode.log:[2024-04-12T22:54:01.303438Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWS7cM4z2LmQVromi9                                                  9W2bWABv6RBfFuwCKHstmgskFzBs - a150acae7beb24eff6a44f4e13fb683f4da275a0d6dca936cc2434041a1ed5d2) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode42/safenode.log:[2024-04-12T22:58:59.088182Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA3XyJh9Pn7CZoiY7                                                  7TH3LihTpso1Wf249ZkVJr7GvWDV - a17d5f9aceb08dc64a3caef3020b869583c915719fc3f9553de8fb083d0e7953) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode43/safenode.log:[2024-04-12T22:58:06.480372Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWHkqHhDF8QjDJnMHB                                                  HYf7K5tAaCRS4iiDFNweA88sRBrQ - fde4efdb88d7faba1dc341d4b1dd02b80c59e049a81251342db65d589f9cd4d7) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode43/safenode.log:[2024-04-12T22:58:22.617099Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWGFyYUMNvXeZiZXMk                                                  nFMRxn9RDeGKJZ4ttT29fQnrSeTy - fd0334743a8bb34b3b1e87bc7c2652590ff371c42dab3cdfb26fa84a3e5b35a2) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode43/safenode.log:[2024-04-12T22:58:23.137729Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWASDPnsk1LyXxgc25                                                  ubsQ1RtQ85BpuiVypbNcA7QPt3bC - fc963c59c2e3f1bec6efe9392cebf98ed1a23db04d55b8da0590a3f162d4bbde) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode45/safenode.log:[2024-04-13T03:35:36.723268Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWG5QZFonnDrTN7A8a                                                  Lukbt7tHDJRjXFEy1Mi7sYqSYnsx - 7a21e5c908d56923bffaa1901a4fc8068a939d420baf2f35f0a042c204a3ada9) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode45/safenode.log:[2024-04-13T03:36:18.919948Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWP6ixDxqgub6rMGsT                                                  gjajn45Y8WwBfDWLPmhEccjmM47p - 7a302c85d09154435d23d83a0d4ebcdaaa892010b92e0697af71256b5b2388ff) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode45/safenode.log:[2024-04-13T03:37:32.742183Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWEfCMs6CyVum2FCqJ                                                  QT47LKascDkMdUuJhxzsnXtmqsw2 - 78f1012ee863a6e9df4435a56925a426d9fb15e4f3a3bfeda5450b30a8168eae) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode45/safenode.log:[2024-04-13T03:38:08.351161Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWS1FL6Y39HGQ6g9ZY                                                  6vyNJKXfoWWoZHsFyotDgHU2QQH4 - 7b1e157c72ab2407ba522e14f62198d7f205d00941865983ccd4aa482f3e1e4b) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode46/safenode.log.20240413T094403:[2024-04-12T23:10:13.868524Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  NpVEECd7jVpsPP5LSQxoHSuoZPzDE4zdzeKiwJc3SWfw - 9d50869aaed99bf6696710aa66bcb22ab4618bfd6b1ab8ff98c6c24e2ce32e89) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode46/safenode.log.20240413T094403:[2024-04-12T23:10:14.035956Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  QweX7MsBXqX3Chj4tczBce23k1jgrTo53hNjD8B4uvKN - 9d469bfd27d212244fdb093cc0e8b82dfbe1ccdea14e3da34a9a6e689994e600) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode46/safenode.log.20240413T094403:[2024-04-12T23:10:36.757076Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  QqAvyXHuY6zNKojxdbHHPMZgaJyFDFQoLMWV1fD5gryJ - 9d4b0a432c564c860c8a4d7f5f8afedb27adeb3c2976375b4578e7810d0353ce) consider us as BAD, due to "FailedChu                                                  nkProofCheck".
/var/log/safenode/safenode46/safenode.log.20240413T094403:[2024-04-12T23:32:51.042277Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  AnJFk1eagoNcWxgEuoHzGGezf7QkgNEAhmtWnrhBnW8w - 9c9db0adf7faa00d4c6ab2afefafd69cc08f4d75649a4f0449fae0f0f9ac5c8c) consider us as BAD, due to "Replicati                                                  onFailure".
/var/log/safenode/safenode46/safenode.log.20240413T094403:[2024-04-13T09:43:29.863132Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  JD4F83hjEcCfxgaLLse6khPjJSeRts7s1rUTKZmSTkKh - 9c225d91b3debfe0f3c43510562caddd20628a55d555b7675604cb473f089bf1) consider us as BAD, due to "Replicati                                                  onFailure".
/var/log/safenode/safenode47/safenode.log:[2024-04-12T23:14:21.340593Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWG4SNhh5GYNCLVVMR                                                  oG2gBi5YswQJ8WArwzghmBXBGTfw - cf213bd201c144acb95f784361e0b619fb7dee1ee4396dfec57588038587170a) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode47/safenode.log:[2024-04-12T23:14:21.340735Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWS5eJBAqGhEH369yu                                                  P4ekeMJAV5UVVYmUAzKJo72XvNTs - cf346db1e6fe8bb6b03781f7cb2635b3bab9d3b7ae60de5db1d90dadaf82634b) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode47/safenode.log:[2024-04-12T23:14:31.058058Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWAugVPqCPbK1HFXpN                                                  i8Wnk2Z9fEGNaYiuFyNF4hn4EP13 - cfb5875ab31896f728d2bdf3b9ad7d89760cafbaa0e935476285122fe4e3e129) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode48/safenode.log:[2024-04-12T23:18:30.823442Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWH953R48SBqsP42hR                                                  dvJbCK195KnY8MDhrwrhvsDW55Rg - 9745cd9707830bbe3472a90c9df53098de95f87aa82659383434270468dc3e72) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode48/safenode.log:[2024-04-12T23:18:30.941452Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWE1NPbxvPW1bwd9dV                                                  jkPbrqjdVLKc2F8RyZGyuctUoVqu - 977a38a9d52f62ea49404bd70797fbc2a01d0f3c6f5522998f332e93a880c82e) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode48/safenode.log:[2024-04-13T03:37:24.262777Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLbSmxBGEkkZVyUkz                                                  jq5FHGQNLxEiST8DgmH59fydtnFa - 979100085ee027825c162fc8f3abbdc014c2e71486b5e5b5b797f4e3bdc8fcb5) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode48/safenode.log:[2024-04-13T03:37:30.154823Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWLcwZEBgJY1FZWnv4                                                  XDi6PAawWawpNRkSZ19Eg7xDMBfH - 97251506c1aec2a7f3d5e3ae4984e4d590e993fd7af55969581ff28dfe06d3d9) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode48/safenode.log:[2024-04-13T03:37:47.748157Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNZ62fLyN4yCyN5fh                                                  bHtk3zNsgwzsN6Vm18wWqWKqyg7A - 97c9b708c12252a40a11598c5b0248b548022e2f718296441e646ae20e00498b) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode48/safenode.log:[2024-04-13T03:39:58.803573Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWKduhwjkobcUY9GSS                                                  gDhUD8HJPxJ396219hPTkvVo1gNU - 96d5637ac13b6a6b8fee651e363245104e953b36b2d9092136b1a01a6181e4c8) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode49/safenode.log:[2024-04-12T23:22:36.329325Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW9rnHszicwSDy4bhr                                                  HMWoVTG1mKJ1V89i93JxZ9r9qer2 - c48ff567f49fd71689a09c54719e835c32ef7b18c1b3d6381dff5d743915de6d) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode49/safenode.log:[2024-04-12T23:23:05.448809Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWS4yHGZPcshVnDDoG                                                  ziyT7oDMT1yaJ9otk1MubFJoEuyr - c4403894a5dea301df0687350f947a7cd94a95d76cc2b4b88eb19acb4f33d230) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode49/safenode.log:[2024-04-12T23:23:05.449249Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWSBE8C5cuA9SerkPZ                                                  bm8aSHLsQUo34yyQY7m3v1DkiY3U - c4ffda4b19cab32feb5b67eac4a1a5d758d75b6f12cac52d13a7f79f6985237e) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode49/safenode.log:[2024-04-12T23:23:08.817631Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWFfGjYc2XS8aVA6ic                                                  qrXkJbETqDTecjgP81isiQtQvMhD - c459a047a6f9a3b0e47dded085109f359ede5653feb72b67948b787f86383313) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode49/safenode.log:[2024-04-12T23:23:57.494936Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA13ayNjZiL52wEhN                                                  f6XZRnAgpVLk2hnifbEswQ8kGvb6 - c46d3fa90a36f6bd311e9727d48b899807780ac81a30dd15ddfe92d73f0b8179) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode4/safenode.log:[2024-04-12T20:19:09.266046Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWNyLNtHbijnwQCfMTJ                                                  WxksjqFREgCWt8Nz6Ehatq39uQi - 137bc1d28c047c2f2d4c49308744a4bd2a06614db044adf1130cbca75823b711) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode50/safenode.log.20240413T035602:[2024-04-12T23:26:55.356763Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW                                                  DF3Y27MqUzPABD7FN4iw3V9eFTRsWbYYa6GMae1DdQxN - 0d3cc9538ba72f9b67fa44d1a092851a9f8f2f0ea37c1ffafff1ed30cffe0d28) consider us as BAD, due to "FailedChu                                                  nkProofCheck".

aatonnomicc · April 13, 2024, 9:54am

had to split it due to amount of characters in the message

/var/log/safenode/safenode5/safenode.log:[2024-04-12T20:23:17.663487Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWQg3caciqEtTRWZmJc                                                  UJXWF1Phw5MrCvVeFkUsaRmTdpC - 7b5bc0449d473cfef52e224c51af5cacbf17c9bf4f947776bc7651410f170dc1) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode5/safenode.log:[2024-04-12T20:23:25.494295Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWCrHmewWJmFArsE42D                                                  p4KTKSi2nAG15k6anBmoUGXHHLg - 7b49cd6a28818892fcf8ea4e9d5c89e08b615ddafd8364d0b0d5c1995922dbc0) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode5/safenode.log:[2024-04-12T22:47:46.675088Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWJ8tsiZrRMQE958UMk                                                  4ErqqJbx9zNu2pzt5xHtVtPF2TX - 7a07d85291be5cce5a9e567b6bfb62ff349df494614e2621f6522ab0712fb4f4) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode5/safenode.log:[2024-04-12T23:20:09.692960Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWP6ixDxqgub6rMGsTg                                                  jajn45Y8WwBfDWLPmhEccjmM47p - 7a302c85d09154435d23d83a0d4ebcdaaa892010b92e0697af71256b5b2388ff) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode5/safenode.log:[2024-04-12T23:24:59.614287Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooW9zPfzBanAUxJNrcwo                                                  SkCDh9YJFfu9i3P7xVSZYQSC36m - 7aa7061717c2ed8cc41fd0ebf5d9fcc8166da54038c9d4f7080aec6ae051481e) consider us as BAD, due to "FailedChunkProofCheck".
/var/log/safenode/safenode6/safenode.log:[2024-04-13T03:38:43.867737Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWFN6CepbKA4NadTDaQ                                                  pHsXPztYP2YA3spPJ1kNEkrajtG - 44d92c760a24a797fb9471bad91cabfed84390b004bc9fef9051e93236b14cbd) consider us as BAD, due to "ReplicationFailure".
/var/log/safenode/safenode6/safenode.log.20240412T204307:[2024-04-12T20:27:19.252059Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWJ                                                  Ui7i7KxWjynXxM4GjMNv7AWRaq2tQWEQAY9joK3coUg - 457757668eaa620397e5852c8941b88c6ed284cc6f55af2a356be310f61be684) consider us as BAD, due to "FailedChun                                                  kProofCheck".
/var/log/safenode/safenode6/safenode.log.20240412T204307:[2024-04-12T20:27:24.009850Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWA                                                  bt5n9pNw1T6zSSu3ymxjx7WS4yupHQbNVdSdmhLKFe4 - 45fc09b870ef49016e81694f0fddc5879a41e45b56c4464d50ab2c569db3983e) consider us as BAD, due to "FailedChun                                                  kProofCheck".
/var/log/safenode/safenode7/safenode.log.20240413T050246:[2024-04-12T20:32:06.091353Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWB                                                  GzLjn8bot2igsLJ2xkcZBoxobAinGV9QhGLGHqcvZyH - f69025b1c98ffc7b7946cb81d6191b0dffcbcd55e840d9871f4a4bb80bfcf4be) consider us as BAD, due to "FailedChun                                                  kProofCheck".
/var/log/safenode/safenode7/safenode.log.20240413T050246:[2024-04-12T22:30:56.072929Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWE                                                  LMdkSNjGmqJFoVUiJsuto6ge3Rn96MhPzk3yhUoNym2 - f0c0088e61ff8f47df08fbf72ee66cfb40c870894e445308429e7763586070ba) consider us as BAD, due to "Replicatio                                                  nFailure".
/var/log/safenode/safenode7/safenode.log.20240413T050246:[2024-04-12T23:20:22.127222Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWR                                                  Q1jpAaGrhgTet5C7L6sUA9FcboVx68iVYmW9qNGUS5k - f35566d6a678c6e8b4edf113b6f94ad1c18f7f285c2daedfcec5933ec0a81272) consider us as BAD, due to "FailedChun                                                  kProofCheck".
/var/log/safenode/safenode7/safenode.log.20240413T050246:[2024-04-12T23:35:01.942355Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWH                                                  BZEWAVKjBxFP8n667LVZLKbFUs5XHic81MJcuFzVsXR - f3834cb5c92699b1c1df9ab6cfa028d73a39a77809dc902a6ed5d9b36557f165) consider us as BAD, due to "FailedChun                                                  kProofCheck".
/var/log/safenode/safenode9/safenode.log.20240413T045345:[2024-04-12T23:18:59.579848Z WARN sn_networking::event] Peer NetworkAddress::PeerId(12D3KooWC                                                  djLqnCco2GLgnzEk6Yt7K3uauU68JW9N4AkTWSpo1As - d94a576f2958151907f094bc57bc9c1e7dc3038e0d3f8c17e9ae5d21132f03c7) consider us as BAD, due to "Replicatio                                                  nFailure".

aatonnomicc · April 13, 2024, 10:26am

guessing this is down to the current state of the network just started two nodes with a 15 min interval and one was shunned pretty much as soon as it started.

happybeing · April 13, 2024, 10:45am

Great thanks. I’ll fix vdash to show the reason tomorrow.

Josh · April 13, 2024, 11:16am

I am away again, been months of working far too hard, nonetheless point being I am not able to get stuck in like you at the moment but I am concerned that bad node detection is causing more problems than it solves.

I have tried to piecemeal bits of information together though. I worry that nodes reporting others as bad causes a cascading effect. But admit I do not understand it well enough to be sure. Looking at all your shunned nodes though is troubling.

happybeing · April 13, 2024, 12:32pm

Obviously there’s a risk that will happen, but in this test it was compounded by having incompatible nodes running, and it’s not clear if or how well that has been cleaned up.

It has though highlighted that ‘bad’ node detection will have a threshold that could well cascade if too many nodes become shunned. I think it will help when they can flag incompatible protocol versions and start again.

BTW I think the word ‘bad’ is misleading, because good nodes may be shunned too. I doubt there are any actual ‘bad’ nodes in this test yet.

Josh · April 13, 2024, 1:39pm

Indeed, so my understanding thus far, correct me where I am wrong.

Close nodes make independent decisions, 3 strikes and you are out.
(Not clear to me if close nodes ask for neighbors opinions)

Far nodes, also 3 strikes you are out but that could be 3 neighbors saying you are dodgy?

Perfection 100% of the time is a big ask, so are they keeping time? Assume not, if not, it can’t be that hard to get 3 ticks and inevitably binned.

Do you see it differently?

happybeing · April 13, 2024, 2:03pm

I haven’t looked at the algorithm. My take is that these are experiments and Autonomi are trying out ideas that are bound to fail most of the time to begin with. I get the impression that some expect things to work well right away and to only need tweaking, and so when things degrade badly they get worried.

I’m not laying you’re doing that, but I’m aware that as I’m not deep into thinking about an area I’ll leave it to those who are.

If they appear to be struggling to solve something I’ll be happy to get into it and see if I can contribute, but until then I can be doing things I hope will help in other areas.

Josh · April 13, 2024, 2:09pm

I may be guilty of that, not because testing things out may require multiple iterations to get it right but because there are now dates set that scare me.
None of it worried me before.

rreive · April 13, 2024, 2:35pm

re: the ‘shunning’ algorithm is, as I understand the discussion here ’ three strikes and you are out close groups, then same for far groups’, there are different exception cases here correct?, ie didn’t respond, didn’t respond fast enough, etc… , so perhaps in these “soft” ‘there is something wrong with the communications possibility’ ie the WAN is slowing down for reasons outside the control of the close group, that the algorithm’s first loop logic, should include a ‘cooling off’ period, that is perhaps randomized within a range of time min/max, then the close group allows such nodes failing in these exception use cases to more quickly re-connect, and that the state should be marked as type 'comms issue ? with desc.: ‘cooling off, rejoining group in n seconds’ and, that this exception state is represented in the safenode manager node status list, or the vdash as perhaps some code and/or colour for such node in this exception state (the latter with being mouse over focus of say the state colour change to get the state description in upcoming browser implementation)? Just brainstorming here on possible remedies/fixes/enhancements…

wes · April 13, 2024, 3:44pm

To me it looks like you’re being shunned because you’re failing to provide (or possibly provide in a timely manner?) chunks you’re supposed to be storing. Replication failure I’m not sure.

I just started up 3 more nodes at several minute interval and didn’t get any reports of being a bad node. My logs were flooded with calling out other nodes as being bad almost instantly though.

I had to run out for a bit, but I’ll do some digging on the “why” my new nodes called out so many so fast and post my findings.

happybeing · April 13, 2024, 3:48pm

Haha. Well those dates are challenging as they say. I really wouldn’t want to get onto that, but why worry?

Do you remember Mark Rylance’s character in Bridge of Spies whenever Tom Hanks asked him if he was worried, with the very real prospect he was going to be executed?

I began thinking about him after I replied here, and then decided it would have been a way better film with somebody other than Hanks, and that many of his films would have been better with somebody else… except Road to Perdition. But Castaway… how about Christian Bale?

I decided it was Hollywood’s fault rather than Hanks’ and went on to ponder the good movies with such stars, such as Falling Down. But I digress…

Worried?

Would it help?

Josh · April 13, 2024, 4:01pm

No it doesn’t help, yet we do it anyway.

I take comfort in you thinking that the dates are challenging too!

aatonnomicc · April 13, 2024, 4:45pm

I was using the build from the main repo so may have had an issue with that as well.
I have just scraped everything all nodes are now off line.

ill have a go starting some nodes on a single system with safe node version 0.105.6

how is your cpu usage and how many nodes you running ?

wes · April 13, 2024, 7:30pm

I’m up to 120 or so nodes running on 10 cores, 10 gigs ram and plenty of space.
I’m now using the procstat telegraf plugin so I get CPU samples every 30 seconds for every node, averaged at grafana to 1 minute. Startup of a node does seem relatively CPU intensive.

I want to play with alpine, so I may be spinning up a bunch of new nodes soon and can see what a mass fresh startup looks like.

aatonnomicc · April 13, 2024, 7:41pm

maybe we move back to the test net thread to keep it all in there ?

neo · April 13, 2024, 10:52pm

I was told the nodes time out strikes. 10 minutes or something like that.

Time in the protocols is not there but timing within the node itself is there in one form or another. How do you think the nodes know when the neighbour did not respond fast enough, yes a timeout. Protocols in their error recovery steps within the device are using time to know when the response should have arrived, how long to wait before a retry, etc. But network, time is not there, internode time is not there, time of day anywhere is not there, but timeouts is there within each node.

Josh · April 21, 2024, 10:32am

I still don’t grok the system, really wish there was a thorough doc on it.

Oxford dictionary: shunned = persistently avoided, ignored, or rejected.

Topic		Replies	Views
RPI 4: Is this thing even on? Support	12	382	August 31, 2024
Update 25th April, 2024 Updates	14	1142	May 2, 2024
Node status [error] definitions Support	1	62	September 15, 2024
Back again after another extended (non-testing) absence . . can't connect . Support	9	92	December 14, 2024
Safenode program built from current github will not join the network (got the keys from logs) Support	6	56	December 1, 2024

Shunned

Related topics