It also contains gzipped logs and what more important - at some point in time logs starts getting deleted (at least it was the case some time ago).
So if after some uptime you need to look what happened during node start - you may realize that logs from the start already deleted.
I think it must be the case that we have changed something that causes much more log output to be produced, because from what I can see, we haven’t altered the configuration.
I’m going to investigate the configuration of the rotation.
2 Likes
Hey @Jadkin , thanks for joining
Interesting. We’re really relying on libp2p here. What sort of timeframe are we talking about all these sessions? (how many connections does that correlate to? can you tell?)
There could be many reasons. I’d wager it’s (insufficient) payment related. The payment logic in this version is still pretty flawed (as mentioned elsewhere above). We’re really not testing uploads so heavily in this iteration so much as node connectivity.
5 Likes
Right now, log directory for my node contains 386 files, 225 MB in total.
List
01.09.2023 18:33 0 safenode.log
31.08.2023 19:55 151 290 safenode.log.20230831T115532.gz
31.08.2023 19:58 200 917 safenode.log.20230831T120033.gz
31.08.2023 20:02 187 931 safenode.log.20230831T120541.gz
31.08.2023 20:05 187 170 safenode.log.20230831T121110.gz
31.08.2023 20:09 187 549 safenode.log.20230831T121635.gz
31.08.2023 20:12 186 523 safenode.log.20230831T122155.gz
31.08.2023 20:16 182 693 safenode.log.20230831T122644.gz
31.08.2023 20:19 196 710 safenode.log.20230831T123129.gz
31.08.2023 20:22 187 189 safenode.log.20230831T123613.gz
31.08.2023 20:25 194 226 safenode.log.20230831T124132.gz
31.08.2023 20:29 185 955 safenode.log.20230831T124632.gz
31.08.2023 20:33 198 666 safenode.log.20230831T125132.gz
31.08.2023 20:37 192 300 safenode.log.20230831T125722.gz
31.08.2023 20:41 182 227 safenode.log.20230831T130233.gz
31.08.2023 20:46 192 550 safenode.log.20230831T130807.gz
31.08.2023 20:52 183 522 safenode.log.20230831T131315.gz
31.08.2023 20:57 196 288 safenode.log.20230831T131816.gz
31.08.2023 21:01 194 290 safenode.log.20230831T132329.gz
31.08.2023 21:06 191 911 safenode.log.20230831T132827.gz
31.08.2023 21:11 192 622 safenode.log.20230831T133316.gz
31.08.2023 21:16 194 014 safenode.log.20230831T133848.gz
31.08.2023 21:21 185 643 safenode.log.20230831T134416.gz
31.08.2023 21:26 188 893 safenode.log.20230831T134917.gz
31.08.2023 21:31 194 415 safenode.log.20230831T135434.gz
31.08.2023 21:35 192 680 safenode.log.20230831T140005.gz
31.08.2023 21:40 194 464 safenode.log.20230831T140428.gz
31.08.2023 21:45 193 379 safenode.log.20230831T140915.gz
31.08.2023 21:50 190 533 safenode.log.20230831T141439.gz
31.08.2023 21:55 188 824 safenode.log.20230831T141922.gz
31.08.2023 21:59 191 440 safenode.log.20230831T142358.gz
31.08.2023 22:04 189 679 safenode.log.20230831T142836.gz
31.08.2023 22:09 201 960 safenode.log.20230831T143334.gz
31.08.2023 22:13 185 740 safenode.log.20230831T143837.gz
31.08.2023 22:18 188 505 safenode.log.20230831T144309.gz
31.08.2023 22:23 192 631 safenode.log.20230831T144744.gz
31.08.2023 22:28 193 930 safenode.log.20230831T145250.gz
31.08.2023 22:32 192 589 safenode.log.20230831T145813.gz
31.08.2023 22:37 192 723 safenode.log.20230831T150310.gz
31.08.2023 22:41 192 501 safenode.log.20230831T150748.gz
31.08.2023 22:45 201 944 safenode.log.20230831T151248.gz
31.08.2023 22:50 192 298 safenode.log.20230831T151804.gz
31.08.2023 22:55 190 571 safenode.log.20230831T152247.gz
31.08.2023 23:00 194 884 safenode.log.20230831T152737.gz
31.08.2023 23:06 188 594 safenode.log.20230831T153218.gz
31.08.2023 23:11 196 926 safenode.log.20230831T153656.gz
31.08.2023 23:15 192 532 safenode.log.20230831T154146.gz
31.08.2023 23:20 192 639 safenode.log.20230831T154626.gz
31.08.2023 23:25 202 004 safenode.log.20230831T155102.gz
31.08.2023 23:30 193 566 safenode.log.20230831T155608.gz
31.08.2023 23:35 194 666 safenode.log.20230831T160041.gz
31.08.2023 23:39 199 826 safenode.log.20230831T160534.gz
31.08.2023 23:44 195 774 safenode.log.20230831T161017.gz
31.08.2023 23:50 190 562 safenode.log.20230831T161449.gz
31.08.2023 23:54 194 643 safenode.log.20230831T161948.gz
31.08.2023 23:59 192 468 safenode.log.20230831T162455.gz
01.09.2023 00:04 191 183 safenode.log.20230831T162933.gz
01.09.2023 00:08 199 842 safenode.log.20230831T163431.gz
01.09.2023 00:13 193 114 safenode.log.20230831T163941.gz
01.09.2023 00:18 189 229 safenode.log.20230831T164452.gz
01.09.2023 00:23 194 439 safenode.log.20230831T164950.gz
01.09.2023 00:27 194 026 safenode.log.20230831T165446.gz
01.09.2023 00:32 197 754 safenode.log.20230831T165950.gz
01.09.2023 00:37 193 198 safenode.log.20230831T170401.gz
01.09.2023 00:43 202 559 safenode.log.20230831T170924.gz
01.09.2023 00:47 191 866 safenode.log.20230831T171411.gz
01.09.2023 00:52 198 622 safenode.log.20230831T171901.gz
01.09.2023 00:58 197 379 safenode.log.20230831T172330.gz
01.09.2023 01:03 205 202 safenode.log.20230831T172753.gz
01.09.2023 01:07 201 985 safenode.log.20230831T173245.gz
01.09.2023 01:12 191 641 safenode.log.20230831T173746.gz
01.09.2023 01:17 189 437 safenode.log.20230831T174208.gz
01.09.2023 01:22 189 833 safenode.log.20230831T174634.gz
01.09.2023 01:27 196 823 safenode.log.20230831T175111.gz
01.09.2023 01:32 188 550 safenode.log.20230831T175559.gz
01.09.2023 01:37 198 742 safenode.log.20230831T180112.gz
01.09.2023 01:42 194 981 safenode.log.20230831T180643.gz
01.09.2023 01:47 192 314 safenode.log.20230831T181151.gz
01.09.2023 01:51 194 667 safenode.log.20230831T181733.gz
01.09.2023 01:56 193 015 safenode.log.20230831T182234.gz
01.09.2023 02:01 198 688 safenode.log.20230831T182703.gz
01.09.2023 02:06 194 306 safenode.log.20230831T183053.gz
01.09.2023 02:11 199 694 safenode.log.20230831T183503.gz
01.09.2023 02:15 200 951 safenode.log.20230831T183959.gz
01.09.2023 02:20 195 836 safenode.log.20230831T184451.gz
01.09.2023 02:25 194 041 safenode.log.20230831T185046.gz
01.09.2023 02:30 186 742 safenode.log.20230831T185520.gz
01.09.2023 02:35 200 706 safenode.log.20230831T190009.gz
01.09.2023 02:40 197 308 safenode.log.20230831T190457.gz
01.09.2023 02:46 191 094 safenode.log.20230831T190858.gz
01.09.2023 02:51 201 366 safenode.log.20230831T191338.gz
01.09.2023 02:55 195 182 safenode.log.20230831T191838.gz
01.09.2023 03:01 187 517 safenode.log.20230831T192313.gz
01.09.2023 03:06 194 742 safenode.log.20230831T192739.gz
01.09.2023 03:11 191 400 safenode.log.20230831T193127.gz
01.09.2023 03:17 176 499 safenode.log.20230831T193454.gz
01.09.2023 03:22 188 798 safenode.log.20230831T193841.gz
01.09.2023 03:26 189 063 safenode.log.20230831T194217.gz
01.09.2023 03:31 181 451 safenode.log.20230831T194526.gz
01.09.2023 03:36 193 271 safenode.log.20230831T194904.gz
01.09.2023 03:41 197 060 safenode.log.20230831T195200.gz
01.09.2023 03:46 191 895 safenode.log.20230831T195530.gz
01.09.2023 03:50 186 845 safenode.log.20230831T195837.gz
01.09.2023 03:55 189 793 safenode.log.20230831T200228.gz
01.09.2023 03:59 187 235 safenode.log.20230831T200530.gz
01.09.2023 04:04 192 641 safenode.log.20230831T200903.gz
01.09.2023 04:10 192 352 safenode.log.20230831T201232.gz
01.09.2023 04:15 194 241 safenode.log.20230831T201610.gz
01.09.2023 04:20 191 870 safenode.log.20230831T201904.gz
01.09.2023 04:25 194 989 safenode.log.20230831T202230.gz
01.09.2023 04:30 194 002 safenode.log.20230831T202550.gz
01.09.2023 04:35 183 532 safenode.log.20230831T202921.gz
01.09.2023 04:40 190 069 safenode.log.20230831T203323.gz
01.09.2023 04:44 193 230 safenode.log.20230831T203704.gz
01.09.2023 04:49 204 144 safenode.log.20230831T204156.gz
01.09.2023 04:54 188 121 safenode.log.20230831T204646.gz
01.09.2023 04:59 191 547 safenode.log.20230831T205208.gz
01.09.2023 05:04 188 761 safenode.log.20230831T205713.gz
01.09.2023 05:09 195 253 safenode.log.20230831T210139.gz
01.09.2023 05:14 195 688 safenode.log.20230831T210626.gz
01.09.2023 05:19 191 466 safenode.log.20230831T211144.gz
01.09.2023 05:24 188 712 safenode.log.20230831T211628.gz
01.09.2023 05:29 197 475 safenode.log.20230831T212149.gz
01.09.2023 05:34 182 393 safenode.log.20230831T212629.gz
01.09.2023 05:39 202 059 safenode.log.20230831T213103.gz
01.09.2023 05:44 195 300 safenode.log.20230831T213557.gz
01.09.2023 05:49 189 865 safenode.log.20230831T214028.gz
01.09.2023 05:54 192 234 safenode.log.20230831T214518.gz
01.09.2023 05:58 195 569 safenode.log.20230831T215030.gz
01.09.2023 06:03 187 470 safenode.log.20230831T215512.gz
01.09.2023 06:08 191 507 safenode.log.20230831T215944.gz
01.09.2023 06:13 193 159 safenode.log.20230831T220414.gz
01.09.2023 06:18 196 170 safenode.log.20230831T220902.gz
01.09.2023 06:23 186 268 safenode.log.20230831T221357.gz
01.09.2023 06:27 188 826 safenode.log.20230831T221822.gz
01.09.2023 06:32 199 263 safenode.log.20230831T222307.gz
01.09.2023 06:36 191 803 safenode.log.20230831T222759.gz
01.09.2023 06:41 193 721 safenode.log.20230831T223236.gz
01.09.2023 06:46 199 346 safenode.log.20230831T223715.gz
01.09.2023 06:51 192 895 safenode.log.20230831T224135.gz
01.09.2023 06:56 200 885 safenode.log.20230831T224545.gz
01.09.2023 07:01 198 447 safenode.log.20230831T225050.gz
01.09.2023 07:06 192 613 safenode.log.20230831T225547.gz
01.09.2023 07:11 192 969 safenode.log.20230831T230044.gz
01.09.2023 07:15 192 656 safenode.log.20230831T230606.gz
01.09.2023 07:20 190 881 safenode.log.20230831T231123.gz
01.09.2023 07:25 190 494 safenode.log.20230831T231527.gz
01.09.2023 07:30 201 652 safenode.log.20230831T232008.gz
01.09.2023 07:34 194 459 safenode.log.20230831T232512.gz
01.09.2023 07:39 193 300 safenode.log.20230831T233015.gz
01.09.2023 07:43 191 907 safenode.log.20230831T233516.gz
01.09.2023 07:48 192 464 safenode.log.20230831T233957.gz
01.09.2023 07:53 193 320 safenode.log.20230831T234458.gz
01.09.2023 07:58 188 926 safenode.log.20230831T235005.gz
01.09.2023 08:02 189 779 safenode.log.20230831T235438.gz
01.09.2023 08:08 199 216 safenode.log.20230831T235945.gz
01.09.2023 08:13 191 396 safenode.log.20230901T000413.gz
01.09.2023 08:18 201 054 safenode.log.20230901T000850.gz
01.09.2023 08:23 194 848 safenode.log.20230901T001319.gz
01.09.2023 08:28 198 421 safenode.log.20230901T001823.gz
01.09.2023 08:32 191 671 safenode.log.20230901T002301.gz
01.09.2023 08:37 194 106 safenode.log.20230901T002733.gz
01.09.2023 08:42 194 843 safenode.log.20230901T003215.gz
01.09.2023 08:47 204 533 safenode.log.20230901T003719.gz
01.09.2023 08:52 191 771 safenode.log.20230901T004312.gz
01.09.2023 08:57 190 298 safenode.log.20230901T004756.gz
01.09.2023 09:01 193 848 safenode.log.20230901T005255.gz
01.09.2023 09:05 191 602 safenode.log.20230901T005833.gz
01.09.2023 09:10 186 237 safenode.log.20230901T010304.gz
01.09.2023 09:15 200 511 safenode.log.20230901T010744.gz
01.09.2023 09:19 189 650 safenode.log.20230901T011249.gz
01.09.2023 09:24 196 395 safenode.log.20230901T011753.gz
01.09.2023 09:29 193 203 safenode.log.20230901T012234.gz
01.09.2023 09:34 201 435 safenode.log.20230901T012738.gz
01.09.2023 09:39 191 384 safenode.log.20230901T013226.gz
01.09.2023 09:43 200 287 safenode.log.20230901T013727.gz
01.09.2023 09:48 194 351 safenode.log.20230901T014244.gz
01.09.2023 09:52 185 459 safenode.log.20230901T014711.gz
01.09.2023 09:58 199 576 safenode.log.20230901T015143.gz
01.09.2023 10:03 196 454 safenode.log.20230901T015653.gz
01.09.2023 10:09 188 012 safenode.log.20230901T020149.gz
01.09.2023 10:13 194 734 safenode.log.20230901T020620.gz
01.09.2023 10:18 189 852 safenode.log.20230901T021101.gz
01.09.2023 10:23 190 704 safenode.log.20230901T021537.gz
01.09.2023 10:28 199 392 safenode.log.20230901T022038.gz
01.09.2023 10:32 190 789 safenode.log.20230901T022543.gz
01.09.2023 10:37 194 374 safenode.log.20230901T023010.gz
01.09.2023 10:42 195 052 safenode.log.20230901T023520.gz
01.09.2023 10:46 190 441 safenode.log.20230901T024033.gz
01.09.2023 10:51 187 839 safenode.log.20230901T024603.gz
01.09.2023 10:56 190 749 safenode.log.20230901T025105.gz
01.09.2023 11:02 193 316 safenode.log.20230901T025552.gz
01.09.2023 11:06 192 442 safenode.log.20230901T030123.gz
01.09.2023 11:11 186 985 safenode.log.20230901T030636.gz
01.09.2023 11:16 193 734 safenode.log.20230901T031140.gz
01.09.2023 11:20 191 308 safenode.log.20230901T031713.gz
01.09.2023 11:25 192 292 safenode.log.20230901T032225.gz
01.09.2023 11:29 190 153 safenode.log.20230901T032659.gz
01.09.2023 11:35 195 290 safenode.log.20230901T033148.gz
01.09.2023 11:40 195 035 safenode.log.20230901T033627.gz
01.09.2023 11:44 195 991 safenode.log.20230901T034130.gz
01.09.2023 11:49 188 776 safenode.log.20230901T034608.gz
01.09.2023 11:53 192 933 safenode.log.20230901T035042.gz
01.09.2023 11:57 193 240 safenode.log.20230901T035514.gz
01.09.2023 12:02 192 819 safenode.log.20230901T035950.gz
01.09.2023 12:07 193 997 safenode.log.20230901T040449.gz
01.09.2023 12:11 194 449 safenode.log.20230901T041033.gz
01.09.2023 12:16 185 426 safenode.log.20230901T041558.gz
01.09.2023 12:21 192 768 safenode.log.20230901T042052.gz
01.09.2023 12:26 192 075 safenode.log.20230901T042544.gz
01.09.2023 12:31 195 818 safenode.log.20230901T043045.gz
01.09.2023 12:36 188 778 safenode.log.20230901T043528.gz
01.09.2023 12:41 193 799 safenode.log.20230901T044011.gz
01.09.2023 12:46 185 587 safenode.log.20230901T044441.gz
01.09.2023 12:52 193 964 safenode.log.20230901T044944.gz
01.09.2023 12:56 194 981 safenode.log.20230901T045441.gz
01.09.2023 13:01 197 963 safenode.log.20230901T045954.gz
01.09.2023 13:06 182 737 safenode.log.20230901T050429.gz
01.09.2023 13:10 199 867 safenode.log.20230901T050954.gz
01.09.2023 13:15 190 527 safenode.log.20230901T051434.gz
01.09.2023 13:20 199 267 safenode.log.20230901T051933.gz
01.09.2023 13:25 196 653 safenode.log.20230901T052447.gz
01.09.2023 13:31 188 360 safenode.log.20230901T052951.gz
01.09.2023 13:35 192 185 safenode.log.20230901T053428.gz
01.09.2023 13:40 196 480 safenode.log.20230901T053901.gz
01.09.2023 13:44 195 049 safenode.log.20230901T054421.gz
01.09.2023 13:49 187 785 safenode.log.20230901T054911.gz
01.09.2023 13:53 194 264 safenode.log.20230901T055407.gz
01.09.2023 13:58 190 821 safenode.log.20230901T055841.gz
01.09.2023 14:03 197 452 safenode.log.20230901T060324.gz
01.09.2023 14:07 196 529 safenode.log.20230901T060821.gz
01.09.2023 14:12 193 139 safenode.log.20230901T061300.gz
01.09.2023 14:17 195 250 safenode.log.20230901T061827.gz
01.09.2023 14:22 190 815 safenode.log.20230901T062343.gz
01.09.2023 14:28 186 417 safenode.log.20230901T062742.gz
01.09.2023 14:33 197 727 safenode.log.20230901T063216.gz
01.09.2023 14:38 191 850 safenode.log.20230901T063650.gz
01.09.2023 14:43 197 215 safenode.log.20230901T064151.gz
01.09.2023 14:48 193 046 safenode.log.20230901T064649.gz
01.09.2023 14:53 191 103 safenode.log.20230901T065125.gz
01.09.2023 14:57 195 955 safenode.log.20230901T065604.gz
01.09.2023 15:02 187 880 safenode.log.20230901T070106.gz
01.09.2023 15:07 193 103 safenode.log.20230901T070617.gz
01.09.2023 15:11 189 794 safenode.log.20230901T071120.gz
01.09.2023 15:16 192 917 safenode.log.20230901T071536.gz
01.09.2023 15:21 194 237 safenode.log.20230901T072017.gz
01.09.2023 15:25 192 763 safenode.log.20230901T072522.gz
01.09.2023 15:30 189 609 safenode.log.20230901T073002.gz
01.09.2023 15:34 191 608 safenode.log.20230901T073434.gz
01.09.2023 15:39 194 741 safenode.log.20230901T073907.gz
01.09.2023 15:44 193 756 safenode.log.20230901T074340.gz
01.09.2023 15:49 192 408 safenode.log.20230901T074840.gz
01.09.2023 15:53 195 050 safenode.log.20230901T075319.gz
01.09.2023 15:58 197 437 safenode.log.20230901T075821.gz
01.09.2023 16:03 191 932 safenode.log.20230901T080257.gz
01.09.2023 16:08 196 131 safenode.log.20230901T080800.gz
01.09.2023 16:13 193 540 safenode.log.20230901T081331.gz
01.09.2023 16:18 191 097 safenode.log.20230901T081809.gz
01.09.2023 16:22 196 005 safenode.log.20230901T082314.gz
01.09.2023 16:26 190 739 safenode.log.20230901T082801.gz
01.09.2023 16:31 195 757 safenode.log.20230901T083251.gz
01.09.2023 16:37 192 789 safenode.log.20230901T083732.gz
01.09.2023 16:41 196 613 safenode.log.20230901T084235.gz
01.09.2023 16:46 189 320 safenode.log.20230901T084708.gz
01.09.2023 16:51 201 095 safenode.log.20230901T085207.gz
01.09.2023 16:56 196 852 safenode.log.20230901T085717.gz
01.09.2023 17:01 185 552 safenode.log.20230901T090117.gz
01.09.2023 17:06 204 753 safenode.log.20230901T090550.gz
01.09.2023 17:10 190 778 safenode.log.20230901T091033.gz
01.09.2023 17:15 186 789 safenode.log.20230901T091509.gz
01.09.2023 17:21 195 934 safenode.log.20230901T091925.gz
01.09.2023 17:25 197 931 safenode.log.20230901T092410.gz
01.09.2023 17:30 194 219 safenode.log.20230901T092921.gz
01.09.2023 17:35 189 712 safenode.log.20230901T093407.gz
01.09.2023 17:40 194 554 safenode.log.20230901T093903.gz
01.09.2023 17:44 190 856 safenode.log.20230901T094326.gz
01.09.2023 17:49 192 821 safenode.log.20230901T094805.gz
01.09.2023 17:55 202 669 safenode.log.20230901T095253.gz
01.09.2023 18:00 188 770 safenode.log.20230901T095803.gz
01.09.2023 18:04 193 307 safenode.log.20230901T100327.gz
01.09.2023 18:09 195 565 safenode.log.20230901T100912.gz
01.09.2023 18:13 185 323 safenode.log.20230901T101324.gz
01.09.2023 18:18 202 363 safenode.log.20230901T101822.gz
01.09.2023 18:23 190 654 safenode.log.20230901T102331.gz
01.09.2023 18:28 193 251 safenode.log.20230901T102831.gz
01.09.2023 18:33 195 976 safenode.log.20230901T103258.gz
01.09.2023 10:37 1 836 244 safenode.log.20230901T103738
01.09.2023 10:42 1 804 098 safenode.log.20230901T104239
01.09.2023 10:46 1 775 686 safenode.log.20230901T104638
01.09.2023 10:51 1 847 534 safenode.log.20230901T105130
01.09.2023 10:56 1 801 057 safenode.log.20230901T105640
01.09.2023 11:02 1 820 206 safenode.log.20230901T110214
01.09.2023 11:06 1 775 462 safenode.log.20230901T110645
01.09.2023 11:11 1 806 144 safenode.log.20230901T111105
01.09.2023 11:16 1 851 046 safenode.log.20230901T111609
01.09.2023 11:20 1 737 453 safenode.log.20230901T112037
01.09.2023 11:25 1 860 683 safenode.log.20230901T112520
01.09.2023 11:29 1 800 925 safenode.log.20230901T112956
01.09.2023 11:35 1 830 211 safenode.log.20230901T113506
01.09.2023 11:40 1 786 076 safenode.log.20230901T114006
01.09.2023 11:44 1 806 832 safenode.log.20230901T114415
01.09.2023 11:49 1 810 582 safenode.log.20230901T114908
01.09.2023 11:53 1 782 492 safenode.log.20230901T115308
01.09.2023 11:57 1 847 201 safenode.log.20230901T115740
01.09.2023 12:02 1 814 597 safenode.log.20230901T120215
01.09.2023 12:07 1 824 929 safenode.log.20230901T120723
01.09.2023 12:11 1 773 721 safenode.log.20230901T121152
01.09.2023 12:16 1 814 722 safenode.log.20230901T121620
01.09.2023 12:21 1 845 181 safenode.log.20230901T122105
01.09.2023 12:26 1 795 813 safenode.log.20230901T122609
01.09.2023 12:31 1 836 607 safenode.log.20230901T123126
01.09.2023 12:36 1 789 381 safenode.log.20230901T123632
01.09.2023 12:41 1 827 492 safenode.log.20230901T124146
01.09.2023 12:46 1 812 863 safenode.log.20230901T124657
01.09.2023 12:52 1 808 589 safenode.log.20230901T125204
01.09.2023 12:56 1 801 597 safenode.log.20230901T125648
01.09.2023 13:01 1 787 477 safenode.log.20230901T130141
01.09.2023 13:06 1 843 228 safenode.log.20230901T130613
01.09.2023 13:10 1 798 860 safenode.log.20230901T131047
01.09.2023 13:15 1 823 468 safenode.log.20230901T131527
01.09.2023 13:20 1 816 320 safenode.log.20230901T132030
01.09.2023 13:25 1 825 026 safenode.log.20230901T132539
01.09.2023 13:31 1 800 022 safenode.log.20230901T133105
01.09.2023 13:35 1 782 730 safenode.log.20230901T133516
01.09.2023 13:40 1 849 626 safenode.log.20230901T134017
01.09.2023 13:44 1 780 896 safenode.log.20230901T134453
01.09.2023 13:49 1 808 398 safenode.log.20230901T134924
01.09.2023 13:53 1 809 589 safenode.log.20230901T135351
01.09.2023 13:58 1 849 899 safenode.log.20230901T135837
01.09.2023 14:03 1 788 056 safenode.log.20230901T140332
01.09.2023 14:07 1 791 370 safenode.log.20230901T140732
01.09.2023 14:12 1 868 825 safenode.log.20230901T141238
01.09.2023 14:17 1 780 480 safenode.log.20230901T141741
01.09.2023 14:22 1 848 511 safenode.log.20230901T142246
01.09.2023 14:28 1 805 846 safenode.log.20230901T142804
01.09.2023 14:33 1 811 786 safenode.log.20230901T143339
01.09.2023 14:38 1 784 914 safenode.log.20230901T143857
01.09.2023 14:43 1 795 951 safenode.log.20230901T144330
01.09.2023 14:48 1 834 408 safenode.log.20230901T144823
01.09.2023 14:53 1 798 477 safenode.log.20230901T145304
01.09.2023 14:57 1 817 776 safenode.log.20230901T145749
01.09.2023 15:02 1 804 325 safenode.log.20230901T150229
01.09.2023 15:07 1 818 384 safenode.log.20230901T150727
01.09.2023 15:11 1 785 753 safenode.log.20230901T151151
01.09.2023 15:16 1 812 377 safenode.log.20230901T151607
01.09.2023 15:21 1 822 203 safenode.log.20230901T152106
01.09.2023 15:25 1 784 640 safenode.log.20230901T152521
01.09.2023 15:30 1 851 915 safenode.log.20230901T153016
01.09.2023 15:34 1 788 826 safenode.log.20230901T153453
01.09.2023 15:39 1 832 579 safenode.log.20230901T153954
01.09.2023 15:44 1 799 999 safenode.log.20230901T154454
01.09.2023 15:49 1 786 566 safenode.log.20230901T154921
01.09.2023 15:53 1 822 928 safenode.log.20230901T155341
01.09.2023 15:58 1 826 741 safenode.log.20230901T155832
01.09.2023 16:03 1 800 559 safenode.log.20230901T160334
01.09.2023 16:08 1 801 292 safenode.log.20230901T160811
01.09.2023 16:13 1 830 402 safenode.log.20230901T161320
01.09.2023 16:18 1 790 335 safenode.log.20230901T161820
01.09.2023 16:22 1 810 458 safenode.log.20230901T162254
01.09.2023 16:26 1 764 463 safenode.log.20230901T162657
01.09.2023 16:31 1 894 325 safenode.log.20230901T163159
01.09.2023 16:37 1 776 781 safenode.log.20230901T163709
01.09.2023 16:41 1 797 615 safenode.log.20230901T164144
01.09.2023 16:46 1 827 570 safenode.log.20230901T164642
01.09.2023 16:51 1 795 936 safenode.log.20230901T165118
01.09.2023 16:56 1 823 229 safenode.log.20230901T165616
01.09.2023 17:01 1 817 632 safenode.log.20230901T170105
01.09.2023 17:06 1 813 507 safenode.log.20230901T170606
01.09.2023 17:10 1 802 410 safenode.log.20230901T171050
01.09.2023 17:15 1 819 935 safenode.log.20230901T171557
01.09.2023 17:21 1 802 945 safenode.log.20230901T172105
01.09.2023 17:25 1 778 751 safenode.log.20230901T172533
01.09.2023 17:30 1 848 096 safenode.log.20230901T173038
01.09.2023 17:35 1 791 384 safenode.log.20230901T173542
01.09.2023 17:40 1 805 505 safenode.log.20230901T174020
01.09.2023 17:44 1 800 617 safenode.log.20230901T174458
01.09.2023 17:49 1 803 006 safenode.log.20230901T174934
01.09.2023 17:55 1 859 201 safenode.log.20230901T175502
01.09.2023 18:00 1 763 337 safenode.log.20230901T180002
01.09.2023 18:04 1 793 463 safenode.log.20230901T180418
01.09.2023 18:09 1 832 156 safenode.log.20230901T180900
01.09.2023 18:13 1 807 013 safenode.log.20230901T181336
01.09.2023 18:18 1 813 021 safenode.log.20230901T181813
01.09.2023 18:23 1 825 184 safenode.log.20230901T182316
01.09.2023 18:28 1 826 447 safenode.log.20230901T182830
01.09.2023 18:33 1 803 788 safenode.log.20230901T183358
4 Likes
Added some more specific log markers for this: chore(node): add valid put log markers by joshuef · Pull Request #693 · maidsafe/safe_network · GitHub
Hmmm, this might be as good as it gets at the moment. Retrieval is all at the libp2p/kad layer, so we don’t have anythign to log just now (probably we can hook in somewhere down the line). But that is just reading from the store, not necessarily serving a chunk (though I think it practice i mayyyy be 1to1… I’m honestly not sure atm)
2 Likes
The logging is configured to produce a rotation every 5000 lines.
I’ll put in a PR to change it to rotate every 20MB.
7 Likes
I’m seeing some errors I’ve not seen before:-
[2023-09-01T21:40:58.240215Z DEBUG sn_networking] Getting record of f41ff46adf4fd4ca896bfba59653d24470b04e9138fe1c41b6d954f578640022 attempts 26/30
thread 'tokio-runtime-worker' panicked at 'range end index 3 out of range for slice of length 0', sn_protocol/src/storage/header.rs:73:32
This is on an AWS Instance with just 1 safenode running as a test.
It has 89 records so far. It got these within just a few minutes. It did seem to be very active. There are lots of entries still going into the logs. Not any new records for a while now.
3 Likes
That message has in fact changed, although apparently not in safe_network main 
Anyway, I have modified vdash and the latest version will pick up PUTS again by only matching on “Wrote record” rather than “Wrote record to disk”. Thanks again @JPL.
5 Likes
I am having no luck with put of any size, it may explain the lack of activity.
New connections add memory but it seems it is not released once they close, at least not all of it.
The spike below was 2042.
1323 before 1275 after.
The number of connections before and after are similar but mem does not return to what it was before the spike.
5 Likes
That is odd. Were you getting uploads before?
On another AWS Instance I have my script running that uploads batches of 100 1MB files of random data and they are all still uploading fine. I’ve not had any errors in the 9 hours it’s been running and that’s a first so is a good result.
I’ve not tried any verifications though. Hopefully I’ll get to that tomorrow.
2 Likes
I did not try any uploads early in the test, my focus was mainly to see if I got empty nodes.
Will try a different peer in the morning.
2 Likes
Failing miserably from an AWS instance… 
ubuntu@DialNetNodesouthside01:~$ SN_LOG=all safenode --log-output-dest data-dir
Logging to directory: "/home/ubuntu/.local/share/safe/node/12D3KooWGKjaTw5PXq4oZM5E6vLv2YNaZDzcW7MJ3Ejbu4CYmMAh/logs"
Using SN_LOG=all
Node is stopping in 1s... Node log path: /home/ubuntu/.local/share/safe/node/12D3KooWGKjaTw5PXq4oZM5E6vLv2YNaZDzcW7MJ3Ejbu4CYmMAh/logs
Error: We have been determined to be behind a NAT. This means we are not reachable externally by other nodes. In the future, the network will implement relays that allow us to still join the network.
Location:
sn_node/src/bin/safenode/main.rs:318:36
Security groups setup is
1 Like
The SG looks good but the safenode command doesn’t seem to be specifying the port to use so it will use a random one I think and could be anything.
this is the command I used:-
SN_LOG=all $HOME/.local/bin/safenode --log-output-dest=$HOME/.local/share/safe/node/1 --root-dir=$HOME/.local/share/safe/node/1 --port=12000
2 Likes
A small update from my corner:
I’ve been downloading every now and then the files I uploaded on Thursday, totaling 625,4 MB, and it’s been successful every time. Takes about 40 minutes to download them. I haven’t tried uploading recently.
My node from home, behind NAT, is running well, getting chunks, and sending them out. Feels great to have a node running! 
Timestamp: sat 2.9.2023 07.56.04 -0400
Node: 12D3KooWKpB3KuUviu2UxJKLuGXoArPv7CvW5jmtmbVS2md5a7o2
PID: 12319
Memory used: 82.5MB
CPU usage: 3.9%
File descriptors: 1626
IO operations:
rchar: 1894097423
wchar: 18048328887
syscr: 3702152
syscw: 14144170
read_bytes: 307957760
write_bytes: 1591767040
cancelled_write_bytes: 0
Threads: 11
Records: 111
Disk usage: 33MB
EDIT:
I just started uploading a folder of 1.2GB of various files. I first tried to upload a bigger folder, but it was too expensive. I deleted files one by one, until my money was enough. Here is logs of last failure, and then the success:
Summary
topi@topi-HP-ProBook-450-G5:~$ safe files upload A_Safe_test_big
Built with git version: 794fca7 / main / 794fca7
Instantiating a SAFE client...
🔗 Connected to the Network Loaded wallet from "/home/topi/.local/share/safe/client/wallet" with balance Token(99995616156)
Preparing (chunking) files at 'A_Safe_test_big'...
Making payment for 2536 Chunks that belong to 21 file/s.
Error: Transfer error Not enough balance, 99.995616156 available, 161.128382464 required
Caused by:
Not enough balance, 99.995616156 available, 161.128382464 required
Location:
sn_cli/src/subcommands/wallet.rs:305:26
topi@topi-HP-ProBook-450-G5:~$ safe files upload A_Safe_test_big
Built with git version: 794fca7 / main / 794fca7
Instantiating a SAFE client...
🔗 Connected to the Network Loaded wallet from "/home/topi/.local/share/safe/client/wallet" with balance Token(99995616156)
Preparing (chunking) files at 'A_Safe_test_big'...
Making payment for 2323 Chunks that belong to 20 file/s.
Successfully made payment of 35232153600 for 2100 records. (At a cost per record of Token(16777216).)
Successfully stored wallet with cached payment proofs, and new balance 63.287067548.
Successfully paid for storage and generated the proofs. They can now be sent to the storage nodes when uploading paid chunks.
Storing file 'exodus_20.5.10_amd64(1).deb' of 68157532 bytes (135 chunk/s)..
10 Likes
I’ve attempted to download (for the third time) my set of 11 pdf files … and am happily reporting that they all came through 
!!! Third time is the charm.
Cheers 
7 Likes
did you increase your time out ?
1 Like
No, I didn’t make any changes. Not sure why it worked this time. Seems like there may need to be some sort of auto-retry to get all the chunks instead of it just quitting. I expect that such a function probably would have retrieved all chunks on the first try.
4 Likes
So, now we have a very stable testnet, with nodes behind NAT. That’s from “ordinary home” in a sense that my router is just some box provided by my ISP, just like they do for thousands (or maybe hundreds of thousands) other Finns. And my connection is very typical, too.Sure, I still need to do manual port forwarding, but the box, the connection, the ISP… etc. works well. That’s something to appreciate.
Then, lot’s of payment related PR’s have merged recently, with nodes getting rewards. So I’d say we are very, very close.
12 Likes
I am having similar results node memory use is creeping up and I am unable to make any uploads of any size.
root@sp1:~/safe# safe files upload test.txt
Built with git version: 794fca7 / main / 794fca7
Instantiating a SAFE client...
🔗 Connected to the Network Loaded wallet from "/root/.local/share/safe/client/wallet" with balance Token(199999987036)
Preparing (chunking) files at 'test.txt'...
Making payment for 1 Chunks that belong to 1 file/s.
Error: Failed to send tokens due to Network Error Could not retrieve the record after storing it: 61f5a0b3072a979aff8b2ef30dead666d464f17fb18cdb4cace1e5149fc10da9.
Location:
sn_cli/src/subcommands/wallet.rs:305:26
2 Likes