Performance testing upload/download using AntTP

neo · February 25, 2025, 12:36am

Another problem is if people are not setting up their connection correctly.

People maybe using port forwarding setting in the node by specifying ports to the launchpad or antnode without setting up their router, and/or opening the ports in the firewall.

So while the nodes are part of the network participating in churning etc, they cannot respond to quote requests since they are not receiving them (blocked by router, dropping unsolicited packets). Or their firewall has not opened those ports.

I suspect there maybe a lot of that. And the main way around that is to request quotes from a lot more neighbouring nodes (say 20 in parallel) and take the first ones, trying for 10 quotes per chunk favouring the closer ones, and if only 5 in reasonable time then that is enough. This allows for a 4 to one failure rate.

Traktion · February 25, 2025, 10:06am

Shouldn’t such nodes get shunned too? If they can’t respond to requests, surely the network should eject them?

neo · February 25, 2025, 10:11am

Yes, I wasn’t suggesting otherwise. But to pin it on if upgraded yet or not, then there has to be allowances for nodes of different versions to work together without penality. How far different is the question. Minor upgrade, major upgrade?

Also there has to be an allowance or it would mean every upgrade, minor or major, would break connectivity if newly upgraded nodes shun the ones that have yet to upgrade and the the other way around.

There needs to be some sort of allowances, which was my reason for saying something

Jadkin · February 25, 2025, 1:53pm

Have tweaked my K6 script, to pass the datamap.list that Neo has created, and using your sn_httpd as the ant client is slow to get peers and get going…

./k6 run -d30m -u 80 -i 1000 ./k6-neo-datamap.js

Initially its really impressive, ~ 8-10seconds to download the file, then as more user threads start, the duration steadly increases to around 50seconds, before the ant_httpd returns ;

ERROR autonomi::client::data_types::chunk] Error fetching chunk ChunkAddress(4668a9): Network(GetRecordError(RecordNotFound))

Assuming that the nodes serving the chunk get overwhelmed and just give up.

Internet link only at 10-15% utlization, and cpu around 13%, I’ve been able to push the downloads well over the 140Mb/S

K6 Summary

     checks.........................: 92.63% 905 out of 977
     data_received..................: 9.5 GB 21 MB/s
     data_sent......................: 539 kB 1.2 kB/s
     http_req_blocked...............: avg=52.31µs  min=1.56µs   med=4.55µs   max=19.59ms  p(90)=8.56µs  p(95)=148.68µs
     http_req_connecting............: avg=32.07µs  min=0s       med=0s       max=16.5ms   p(90)=0s      p(95)=88.36µs
     http_req_duration..............: avg=10.52s   min=167.96µs med=695.96µs max=1m0s     p(90)=36.13s  p(95)=47.14s
       { expected_response:true }...: avg=8.89s    min=167.96µs med=656.24µs max=59.49s   p(90)=31.57s  p(95)=39.45s
     http_req_failed................: 3.21%  95 out of 2955
     http_req_receiving.............: avg=2.5ms    min=0s       med=69.36µs  max=111.83ms p(90)=5.66ms  p(95)=9.29ms
     http_req_sending...............: avg=110.95µs min=4.59µs   med=14.84µs  max=16.43ms  p(90)=39.82µs p(95)=78.47µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s      p(95)=0s
     http_req_waiting...............: avg=10.52s   min=129.48µs med=589.52µs max=1m0s     p(90)=36.12s  p(95)=47.14s
     http_reqs......................: 2955   6.455401/s
     **iteration_duration.............: avg=32.07s   min=2.45s    med=28.24s   max=1m54s    p(90)=1m1s    p(95)=1m1s**
     iterations.....................: 1000   2.184569/s
     vus............................: 1      min=1          max=80
     vus_max........................: 80     min=80         max=80

.

K6 Output / not complete

INFO[0007] Addr: 8709620be538cc3d1edd9c2c3b554d467dfb690f92315377d27f68b05382fd87, 81-12349-upload.file, 10.00 Mb, 7.24 sec  source=console
INFO[0010] Addr: 77efd739b73eb3329f841d1b769caf1f0121cbca59dca1eafcebfb9ee5902738, 50-12349-upload.file, 10.00 Mb, 10.48 sec  source=console
INFO[0010] Addr: b372014c4f3d908f064722f9429308f6ed96072b4c99f854a0cc19d0401c7999, 224-15384-upload.file, 10.00 Mb, 10.65 sec  source=console
INFO[0012] Addr: 73f7ed391bf034a2e3c714fc95f5ea50e7e30fbba32f4760f6cf90934030d6e7, 12.file, 10.00 Mb, 12.09 sec  source=console
INFO[0013] Addr: 940d1c57172b62402e400ae29891ebc4722f6f4e0986c22587f1920da581f080, 348-20355-upload.file, 10.00 Mb, 13.48 sec  source=console
INFO[0014] Addr: 95dfc5332dc973d404fbeac6673cc5dfad2810747b8e3cf3587f023f9823e4b0, 99-15384-upload.file, 10.00 Mb, 14.78 sec  source=console
INFO[0014] Addr: 95dfc5332dc973d404fbeac6673cc5dfad2810747b8e3cf3587f023f9823e4b0, 99-15384-upload.file, 10.00 Mb, 14.78 sec  source=console
INFO[0015] Addr: c652363e75394bd6d25f29c983a9ab129869926a662854e854d9a3e637199c1c, 8.file, 10.00 Mb, 15.65 sec  source=console
INFO[0017] Addr: 46d2b97891fdc8f4809c561c47abf226883d7cb46b25692b9767b66db10899b6, 9-18281-upload.file, 10.00 Mb, 17.07 sec  source=console
INFO[0017] Addr: 23bf3e37a6c77352f53f73fdf04a83ca829170b70da5928c7d20b256b70869f2, 151-15384-upload.file, 10.00 Mb, 17.23 sec  source=console
INFO[0017] Addr: c324f3c88e360e5246389bd5a317a8ffc8e3141e029aa60bd6eec2c2e69a3433, 51.file, 10.00 Mb, 17.50 sec  source=console
INFO[0018] Addr: 5237c7d43caede2148564058e5d3e6e2cc43641c825677b2b3e9ed5611707eef, 18-12349-upload.file, 10.00 Mb, 18.77 sec  source=console
INFO[0019] Addr: af1a4531bce7bf6f1ae3aff408e9a9635b73f6f9e0c9966d799c08f4fd55d6de, 216-15384-upload.file, 10.00 Mb, 19.59 sec  source=console
INFO[0020] Addr: 602a55d2159f491478bdd49c6a30dc088942d686a396713381ce66a6abc64453, 228-15384-upload.file, 10.00 Mb, 19.96 sec  source=console
INFO[0021] Addr: 7aa437483ea9bf6cd026c838e3001608f9fdf0315bcaf890c5acd5b52ae42364, 12.file, 10.00 Mb, 2.43 sec  source=console
INFO[0022] Addr: 092c4c314a14351effe6da312594927ec5da71fddf53e13eb1c633d3eb6d1eaa, 120-15384-upload.file, 10.00 Mb, 7.41 sec  source=console
INFO[0024] Addr: 35a56c91a932a09fee74ec481fdeebfd5e420b1f8028f7e48f730237cee2943e, 104-15384-upload.file, 10.00 Mb, 23.96 sec  source=console
INFO[0024] Addr: 545f4d3d5ca7e9e82db28136737a128e267c4664a8a44429ebe13d858f3a9429, 266-20355-upload.file, 10.00 Mb, 24.10 sec  source=console
INFO[0024] Addr: 0b2a5cae4ae6abdc726dc768ac2f2c77b999d6cd7c6efc299cee5c606149f546, 13-15384-upload.file, 10.00 Mb, 24.11 sec  source=console
INFO[0024] Addr: fdf38e6dff65b2748ae0efa6e81b658bde14979e6175425d16a81e20f1c90ddf, 107-12349-upload.file, 10.00 Mb, 24.23 sec  source=console
INFO[0025] Addr: 7954cc9f18f4bb070a6834e2aa748df2f01c6cd4c73e6f0f8ca3ddf284673668, 175-15384-upload.file, 10.00 Mb, 25.62 sec  source=console
INFO[0026] Addr: c2e8b9090b7973a7f81ac7618e4082ce6caa81cd614e21b1f44c7020bf7f1f74, 191-15384-upload.file, 10.00 Mb, 26.52 sec  source=console
INFO[0028] Addr: 8917fc150b94844510c1fe3dea0ed446f212e62658cb0c3a79f9e4547953ac23, 108-12349-upload.file, 10.00 Mb, 28.48 sec  source=console
INFO[0028] Addr: b405d5ac0fd15f86331472535dbbda8d4351e99cbc5718e1f3570b33b951cb38, 8.file, 10.00 Mb, 28.91 sec  source=console
INFO[0028] Addr: b405d5ac0fd15f86331472535dbbda8d4351e99cbc5718e1f3570b33b951cb38, 8.file, 10.00 Mb, 28.91 sec  source=console
INFO[0029] Addr: ee0cda630b911497f5d1a00a557f1aa9e6353c995d51016c686482adde2fbb2c, 7-18281-upload.file, 10.00 Mb, 9.74 sec  source=console
INFO[0030] Addr: f061d60bd9c7c3dec25164de2951fd33ae00b72bdd9e4757455f2df4b280aa7e, 12.file, 10.00 Mb, 30.46 sec  source=console
INFO[0031] Addr: 08562f4adf925f5d33e41cebd5a863190a1d7980660a9afa79ffb57e9e3cc762, 80-18281-upload.file, 10.00 Mb, 31.24 sec  source=console
INFO[0031] Addr: 3df1570877680b6a3f7ec77e2a55be3aa9503c61bc5aefaa9f3742caca100b14, 38.file, 10.00 Mb, 21.41 sec  source=console
INFO[0032] Addr: db2e659330d06b1e1b60ad277772e040e521fa262ec2fc11a102ec00a0f7e718, 21.file, 10.00 Mb, 19.31 sec  source=console
INFO[0032] Addr: edf093f1561c5b60bcbe00b97479a00e3286e2a9386abe4484bda9feee449099, 102-15384-upload.file, 10.00 Mb, 17.00 sec  source=console
INFO[0033] Addr: 40b398cf5ba6c44575382c70f9c4cd3eaa9f7218025d9213f6c17e24d1d5b4ac, 55.file, 10.00 Mb, 23.47 sec  source=console
INFO[0033] Addr: 32ad3927f059883e9fb6b7fc7962b41313963d69513e60646083623f4266ff84, 88-12349-upload.file, 10.00 Mb, 25.08 sec  source=console
INFO[0033] Addr: 2fdad7c909d752c2de3d4accd90caa155284e43d06c2d1713e6d731a8d8692b0, 6-15384-upload.file, 10.00 Mb, 20.88 sec  source=console
INFO[0034] Addr: d98de5119c19986ef3d32470f950e7354e350051d307574ed71027b497eb6eb5, 154-20355-upload.file, 10.00 Mb, 22.90 sec  source=console
INFO[0035] Addr: ea84e9aaf4919a2b0e7fa3e825622b160b0593363b2c148cddeba5e039015fbc, 39-15384-upload.file, 10.00 Mb, 35.65 sec  source=console
INFO[0035] Addr: 4291b0873329674267c0a4301d3a8471b7d5a28661d0699cdfc0275dd9cad64a, 15.file, 10.00 Mb, 35.65 sec  source=console
INFO[0035] Addr: 9412a9a73ffd37df7949fe3a49f2f69693a6d5e40d1ef89a1dc4d05421600f34, 36-12349-upload.file, 10.00 Mb, 9.26 sec  source=console
INFO[0036] Addr: caa75eeefce9c8edb964b5c7b6813d45789187cdd9b6f604fc81a07dcf363fbf, 85-15384-upload.file, 10.00 Mb, 6.71 sec  source=console
INFO[0036] Addr: caa75eeefce9c8edb964b5c7b6813d45789187cdd9b6f604fc81a07dcf363fbf, 85-15384-upload.file, 10.00 Mb, 6.73 sec  source=console
INFO[0037] Addr: daa6771dce1a73d6b00361b2bc238f6312d4eb301db359c0cf0b4376984d0c9b, 14.file, 10.00 Mb, 18.06 sec  source=console
INFO[0037] Addr: daa6771dce1a73d6b00361b2bc238f6312d4eb301db359c0cf0b4376984d0c9b, 14.file, 10.00 Mb, 18.07 sec  source=console
INFO[0037] Addr: 8e8c57bd754bce91b82a2ccfbb1745dc7ba3eebc07f7453137d09ac7288bbefe, 55-12349-upload.file, 10.00 Mb, 22.14 sec  source=console
INFO[0038] Addr: 8e8c57bd754bce91b82a2ccfbb1745dc7ba3eebc07f7453137d09ac7288bbefe, 55-12349-upload.file, 10.00 Mb, 23.17 sec  source=console
INFO[0038] Addr: 742af451062969f2816f3dc8ad4205de9a1b04fc15669ae94fdb8f8a816021dd, 88-15384-upload.file, 10.00 Mb, 18.75 sec  source=console
INFO[0038] Addr: 373daba351d250fecee7ff92083db5a8be6fae9a63d3df8797e17847209144fb, 4.file, 10.00 Mb, 4.41 sec  source=console
INFO[0038] Addr: 373daba351d250fecee7ff92083db5a8be6fae9a63d3df8797e17847209144fb, 4.file, 10.00 Mb, 4.42 sec  source=console
INFO[0038] Addr: 4868ea1ff44387374e902e356be2ea3cc2974931935be511e188ee92b3c57a4a, 2.file, 10.00 Mb, 18.72 sec  source=console
INFO[0038] Addr: 4868ea1ff44387374e902e356be2ea3cc2974931935be511e188ee92b3c57a4a, 2.file, 10.00 Mb, 18.73 sec  source=console
INFO[0039] Addr: e2b977afe2a7efcd5963c7ba09b74491a1f13bd89b1fd7ff835f1c71d1f1821a, 152-15384-upload.file, 10.00 Mb, 26.11 sec  source=console
INFO[0039] Addr: 2884668d436095faaeb9c690c5c812bb64bdf8123a6fa00071f27322d35706cf, 31-18281-upload.file, 10.00 Mb, 18.06 sec  source=console
INFO[0039] Addr: b1f027824c56115105f7f985d67c15024dbd9511bcf49d8de4b948e8432538ef, 12-12349-upload.file, 10.00 Mb, 13.38 sec  source=console
INFO[0040] Addr: 05a51dac23f7d25d12f3bafcf2446b8a01ddf3e2fdda07e9d6cdc7b8158fb49e, 56-12349-upload.file, 10.00 Mb, 18.25 sec  source=console
INFO[0040] Addr: 05a51dac23f7d25d12f3bafcf2446b8a01ddf3e2fdda07e9d6cdc7b8158fb49e, 56-12349-upload.file, 10.00 Mb, 15.46 sec  source=console
INFO[0041] Addr: 46c663fd4a37bb1f8d7b6ac5909cd4627a001b25bc3eca13a116328634b82a23, 3.file, 10.00 Mb, 14.44 sec  source=console
INFO[0041] Addr: 1df9012dc450bf51d4942ce24fca51eb04f8f7aded3b013d72bbea5e44bdb7f7, 209-20355-upload.file, 10.00 Mb, 11.27 sec  source=console
INFO[0043] Addr: d98de5119c19986ef3d32470f950e7354e350051d307574ed71027b497eb6eb5, 154-20355-upload.file, 10.00 Mb, 17.86 sec  source=console
INFO[0043] Addr: e8c02933322ae3ce2c1a7015ebe506eedb26c496dfe324ad848fab7ea54bacdb, 214-15384-upload.file, 10.00 Mb, 42.97 sec  source=console
INFO[0043] Addr: e224a2c997749b6a7cbd9bf769de75afba3292433f60a294d25eb6e22aee2759, 41.file, 10.00 Mb, 7.68 sec  source=console
INFO[0043] Addr: 9750046d5e2cb6db6d3d81eaa0f44b909bf9537fa3fee32fcbc9b7e509cf841b, 259-20355-upload.file, 10.00 Mb, 27.64 sec  source=console
INFO[0043] Addr: 86043cf3a42538e08d47efee9ff34340f6630b94fc217e055b773954912a1f25, 33.file, 10.00 Mb, 43.31 sec  source=console
INFO[0043] Addr: ebb1252465360578a71720cbe8b356ad2bc5b5da83bc2eeb55210e398c2e21c7, 18.file, 10.00 Mb, 27.90 sec  source=console
INFO[0043] Addr: ebb1252465360578a71720cbe8b356ad2bc5b5da83bc2eeb55210e398c2e21c7, 18.file, 10.00 Mb, 27.92 sec  source=console
INFO[0043] Addr: f4c2d44a9492cb48e47bba417ec618544d05499dce5d522646905b571fe74cf7, 197-15384-upload.file, 10.00 Mb, 32.36 sec  source=console
INFO[0044] Addr: 55d62c112f8a36985132590ddd9b5e33ffbfba5f50fdf67e95253c7dfca9d843, 62-15384-upload.file, 10.00 Mb, 17.39 sec  source=console
INFO[0044] Addr: b35b71c46e423741e004fc4a32856a4ba1782de829d9bfb3c82b66f0ac9b26d2, 36.file, 10.00 Mb, 7.63 sec  source=console
INFO[0044] Addr: 22da52c3de9fec949d297a2402756e0d1f92adb96221f27d6bb95fe940fc8c07, 150-18281-upload.file, 10.00 Mb, 32.61 sec  source=console
INFO[0044] Addr: ab2208abfdcb2370ead8d6c41c83a33bb04c9c0942f6e9180cd9846b2a64e101, 56.file, 10.00 Mb, 28.30 sec  source=console
INFO[0044] Addr: ab2208abfdcb2370ead8d6c41c83a33bb04c9c0942f6e9180cd9846b2a64e101, 56.file, 10.00 Mb, 28.31 sec  source=console
INFO[0046] Addr: ae79c5ccb6bcfe5cf4cdf145e4dc9d5660d33f69d738b0ddc4822d1bce506f9c, 159-15384-upload.file, 10.00 Mb, 28.28 sec  source=console
INFO[0046] Addr: 42a54079cee206fd41136f909f632f3646412c66bd0a8d67102f9ed8bc9713ad, 63-12349-upload.file, 10.00 Mb, 46.92 sec  source=console
INFO[0047] Addr: 95dfc5332dc973d404fbeac6673cc5dfad2810747b8e3cf3587f023f9823e4b0, 99-15384-upload.file, 10.00 Mb, 11.99 sec  source=console
INFO[0047] Addr: 871364f68f09e23685fd61b45af82404e4078c33d1b34be6718d4a970f2a301a, 81-15384-upload.file, 10.00 Mb, 7.46 sec  source=console
INFO[0047] Addr: 943d2301d4086ac698975b8fdc016ee7e00309803c9f7ab35a1419b24e43ad43, 100-12349-upload.file, 10.00 Mb, 17.90 sec  source=console
INFO[0047] Addr: 2aeae741e32f7d9b028fbd81ef28d5b71b0c52476e3ae9edec2ceed72fa6979b, 51-15384-upload.file, 10.00 Mb, 13.59 sec  source=console
INFO[0047] Addr: 0e0898313bbe413ebae455145aabc762eb87595c877fd5a8e3e2755790225ed8, 8-12349-upload.file, 10.00 Mb, 25.17 sec  source=console
INFO[0047] Addr: 0e0898313bbe413ebae455145aabc762eb87595c877fd5a8e3e2755790225ed8, 8-12349-upload.file, 10.00 Mb, 25.18 sec  source=console
INFO[0048] Addr: ea6bc003929e8ecb63797f1b0fcf2ff647b644c6e89a23c08aade67f6f10ca07, 150-15384-upload.file, 10.00 Mb, 22.83 sec  source=console
INFO[0048] Addr: 9051412a1a1aa827e4dfb15da8c013973a79d8f6246f5318049c214a2b41c209, 66-15384-upload.file, 10.00 Mb, 16.53 sec  source=console
INFO[0048] Addr: 2b5c1d63083dab3fff4d14fb6953d43c6506c8bd6bdf0ea3d5d9e91ab8daa92a, 13.file, 10.00 Mb, 13.87 sec  source=console
INFO[0049] Addr: a3c3fc5d77dad08865193bf1e7c607ec0dceac134d7b8005a8e7d2f9edcb5a7b, 118.file, 10.00 Mb, 6.70 sec  source=console
INFO[0049] Addr: 7fe3fd1aaf3b8479c71c3b4f9cd7eb5d1db0d23d6181d1404cc117d808195212, 76-15384-upload.file, 10.00 Mb, 49.74 sec  source=console
INFO[0049] Addr: 509abf958f8c222e4ce1eca728a2b0b3383f0d1adc9590baf2e219a386c0fc30, 13.file, 10.00 Mb, 23.03 sec  source=console
INFO[0050] Addr: 6396b83260d505acec15a4cf2652bf55447a62fcf6ac792ee5a732ae9527b374, 78-18281-upload.file, 10.00 Mb, 35.48 sec  source=console
INFO[0051] Addr: 6b6e39aed3db3a7c910419e9aa2e7e45f88f404bd68ee3b61474f4bc8644ae6b, 53-18281-upload.file, 10.00 Mb, 23.89 sec  source=console
INFO[0051] Addr: c43ed022873c82ae60bb5d9af2b5c08aa8c23bdb8e1223c96ea1d43c2970eab2, 147-15384-upload.file, 10.00 Mb, 32.85 sec  source=console
INFO[0051] Addr: f85b3adfada5b67221d34d5ce549328157ec3c3d89c3e8181f0ae8d9a484a5a9, 115-15384-upload.file, 10.00 Mb, 13.53 sec  source=console
INFO[0052] Addr: f079312cf0869d5f1846e80a3d3079bc6b8407173c3133492510c975aaf39b29, 19.file, 10.00 Mb, 29.17 sec  source=console
INFO[0052] Addr: eb6b44c392ef48ca11c964f46f9b8197adc4be1cdd27e45bc18d5d02cc5e5f03, 80.file, 10.00 Mb, 37.39 sec  source=console
INFO[0052] Addr: ec71455687e3d51ea2106a4caa474b24c9c0506581bc99203c8b357dd496f28e, 38-12349-upload.file, 10.00 Mb, 18.32 sec  source=console
INFO[0053] Addr: 503ba8dbb7695c24461e45e07ccb74819ddae46b0d354f3f855fc7eb3cbe9581, 233-15384-upload.file, 10.00 Mb, 33.45 sec  source=console
INFO[0054] Addr: f9d0c3b36bc3e7e2036990dc81b554e32a13e4612af603e3c71d6958266d9edd, 14-12349-upload.file, 10.00 Mb, 16.87 sec  source=console
INFO[0054] Addr: 8917fc150b94844510c1fe3dea0ed446f212e62658cb0c3a79f9e4547953ac23, 108-12349-upload.file, 10.00 Mb, 29.76 sec  source=console
INFO[0056] Addr: 6d6a26187bcd67714f6d6b8bed569cb43ba802b3db765f122a0ffce32813b5d6, 1.file, 10.00 Mb, 12.00 sec  source=console
INFO[0056] Addr: bde366c8588fd22e88a28be063954bd730de5eb8a4de201010fb224013c463ba, 53-15384-upload.file, 10.00 Mb, 26.12 sec  source=console
INFO[0056] Addr: d95a5d6546b74793d59d0597462f98a401317a8b117a9eb6c45c5a07a152756e, 52.file, 10.00 Mb, 23.77 sec  source=console
INFO[0056] Addr: d95a5d6546b74793d59d0597462f98a401317a8b117a9eb6c45c5a07a152756e, 52.file, 10.00 Mb, 36.42 sec  source=console
INFO[0058] Addr: 4092b6e947088953f4a84381e880b9d391740c5c08b37c00585d901893af8463, 73-15384-upload.file, 10.00 Mb, 11.89 sec  source=console
INFO[0058] Addr: 9deb01b592a32de51d6c1ab280421d6529ff3f7fb7c96b0afa595113ffd56931, 34-15384-upload.file, 10.00 Mb, 2.12 sec  source=console
INFO[0058] Addr: 9deb01b592a32de51d6c1ab280421d6529ff3f7fb7c96b0afa595113ffd56931, 34-15384-upload.file, 10.00 Mb, 2.13 sec  source=console
INFO[0059] Addr: 210e72dba7f1b7a0f50f92dc7512cf717b9892f8cd03ac73097dabdddf6d5ce4, 7.file, 10.00 Mb, 6.43 sec  source=console
INFO[0059] Addr: c5b5c415e2a3241e8f44af00407a9dbc9b8d172ad81cd4cc08519031f6c55055, 44-12349-upload.file, 10.00 Mb, 8.38 sec  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/7ebc383f34adcaebcf0c214f2b491c49d434878acbfdc5e64fe9c89797eb2564\": request timeout"
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/1123bd4b79e6f9d3a161a85539be9351ef25273f4d6b7466f6016f5cd4cfa8ff\": request timeout"
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/bb1c51e56c56e4bcef5ea88c88d38dc06cae82d8ccde9b10348f73d8c3e2ba16\": request timeout"
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/b6b997ef0e3d5ad5edd2b7067e2fbe72ea0d08eb5b3c8f7f7ad8146527f9b65b\": request timeout"
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/a71f24af15e68240bc55fc8b55448baa77b7357c4a27007b5292c9c7b1bf1c40\": request timeout"
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/7ebc383f34adcaebcf0c214f2b491c49d434878acbfdc5e64fe9c89797eb2564\": request timeout"
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/f48beb601dac099a11aab31178d491f59a4d3af81c6df8da7a0891539cc239bb\": request timeout"
ERRO[0060] Error: Cannot read property 'match' of undefined or null  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/10aba9ea7fc87e73678b45e8d01d0b6a5e53e7fcc20b271e4a7a92e1ee986f15/17-12349-upload.file\": request timeout"
ERRO[0060] Name: 17-12349-upload.file, Duration: 60.00 seconds, Download Size: 0 bytes - Error: Download size is less than 1KB  source=console
WARN[0060] Request Failed                                error="Get \"http://localhost:8080/9616630d105c6201d119f6c292f9746139cc494b8242175aa00827a94a0ecc41/28.file\": request timeout"
ERRO[0060] Name: 28.file, Duration: 60.00 seconds, Download Size: 0 bytes - Error: Download size is less than 1KB  source=console
INFO[0061] Addr: dea8ecb2719ebd05a78e23898fdb0d9cd8f157b23d96a8c4b917798c44a8600c, 28-12349-upload.file, 10.00 Mb, 27.74 sec  source=console

.

k6-neo-datamap.js

import { sleep } from 'k6';
import http from 'k6/http';
import { SharedArray } from 'k6/data';
import { check } from 'k6';

const csvData = new SharedArray('data', function () {
    return open('./datamap.list').split('\n').filter(line => !line.startsWith('#') && line.trim() !== '').slice(1);
});

//need to be running an instance on ant_httpd
const SERVER = "http://localhost:8080/";

function getRandomRow() {
    const randomIndex = Math.floor(Math.random() * csvData.length);
    return csvData[randomIndex].split(',');
}

function extractFileName(responseBody) {
    const regex = /<a href="([^"]+)"/;
    const match = responseBody.match(regex);
    return match ? match[1] : null;
}

export default function () {
    try {
        const row = getRandomRow();
        const address = row[0];

        let url = `${SERVER}${address}`;
        let response = http.get(url);

        const name = extractFileName(response.body);
        if (!name || name.length < 3) {
            throw new Error("File name is empty or less than 3 characters");
        }

        url = `${SERVER}${address}/${name}`;

        const start = new Date().getTime();
        response = http.get(url);
        const end = new Date().getTime();
        const duration = (end - start) / 1000;
        const downloadSize = parseInt(response.headers['Content-Length']) || 0;
        const downloadSizeMB = downloadSize / 1048576;

        if (downloadSize < 1024) {
            console.error(`Name: ${name}, Duration: ${duration.toFixed(2)} seconds, Download Size: ${downloadSize} bytes - Error: Download size is less than 1KB`);
            response.status = 501;
        }

        if (response.status !== 200 && response.status !== 501) {
            throw new Error(`Request failed with status: ${response.status}`);
        }

        check(response, {
            'status is 200': (r) => r.status === 200,
        });

        if (response.status === 200) {
            console.log(`Addr: ${address}, ${name}, ${downloadSizeMB.toFixed(2)} Mb, ${duration.toFixed(2)} sec`);
        }

        sleep(1);
    } catch (error) {
        console.error(`Error: ${error.message}`);
    }
}

neo · February 25, 2025, 9:52pm

It is also possible that for one of more files there is some errors in uploading where a chunk was not successfully uploaded. The upload was using the -x option to speed things up, but has the downside that a chunk might not be successfully stored

Be good to know which file so I can remove it from the list if its a problem. My download script cares not if a download error occurs

Traktion · February 25, 2025, 10:47pm

Those are pretty impressive numbers again! Much higher throughput than what I’ve seen for a while.

Are you still getting the same figures for the other test file with just the images?

I’m still finding the network pretty sluggish for most downloads compared to the final test network. I’m looking forward to the new release later in the week.

Traktion · April 2, 2025, 9:34pm

So, with AntTP running 0.4.3 on localhost, I’m getting some much faster numbers again. Still not as fast as the test nets, but it’s starting to get close again…

$ cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js 
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
  //http.get('http://vader.lan/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/st-patrick-monument.mp4');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 730 MB 1.2 MB/s
     data_sent......................: 292 kB 463 B/s
     dropped_iterations.............: 790    1.253966/s
     http_req_blocked...............: avg=12.16µs min=2.12µs   med=5.04µs   max=1.07ms   p(90)=10.05µs p(95)=13.93µs
     http_req_connecting............: avg=1.51µs  min=0s       med=0s       max=433.56µs p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=3.76s   min=928.15ms med=2.02s    max=17.56s   p(90)=10.82s  p(95)=12.04s 
       { expected_response:true }...: avg=3.76s   min=928.15ms med=2.02s    max=17.56s   p(90)=10.82s  p(95)=12.04s 
     http_req_failed................: 0.00%  0 out of 1670
     http_req_receiving.............: avg=1.3ms   min=32.45µs  med=486.37µs max=13.79ms  p(90)=3.91ms  p(95)=5.13ms 
     http_req_sending...............: avg=37.67µs min=7.04µs   med=16.43µs  max=2.42ms   p(90)=38.06µs p(95)=79.57µs
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=3.76s   min=927.95ms med=2.02s    max=17.56s   p(90)=10.81s  p(95)=12.04s 
     http_reqs......................: 1670   2.650788/s
     iteration_duration.............: avg=29.72s  min=18.22s   med=29.7s    max=53.32s   p(90)=38.44s  p(95)=46.84s 
     iterations.....................: 200    0.31746/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m30.0s), 00/10 VUs, 200 complete and 10 interrupted iterations
default ✗ [======>-------------------------------] 10 VUs  10m30.0s/10m0s  0200/1000 shared iters

Traktion · April 3, 2025, 2:42pm

EDIT: I thought it was speeding up, but I had a different test run in my head! ha! Pretty similar to yesterday, actually.

t$ date; cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js 
Thu  3 Apr 15:29:11 BST 2025
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
  //http.get('http://vader.lan/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/st-patrick-monument.mp4');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 558 MB 902 kB/s
     data_sent......................: 222 kB 359 B/s
     dropped_iterations.............: 840    1.35687/s
     http_req_blocked...............: avg=12.36µs min=1.66µs  med=4.43µs   max=2.33ms   p(90)=9.09µs  p(95)=12.68µs
     http_req_connecting............: avg=2.72µs  min=0s      med=0s       max=538.34µs p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=4.83s   min=1.09s   med=2.63s    max=18.39s   p(90)=11.93s  p(95)=12.84s 
       { expected_response:true }...: avg=4.83s   min=1.09s   med=2.63s    max=18.39s   p(90)=11.93s  p(95)=12.84s 
     http_req_failed................: 0.00%  0 out of 1280
     http_req_receiving.............: avg=1.24ms  min=31.38µs med=484.73µs max=16.3ms   p(90)=3.53ms  p(95)=4.64ms 
     http_req_sending...............: avg=41.67µs min=5.64µs  med=14.45µs  max=1.95ms   p(90)=36.36µs p(95)=68.94µs
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s       max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=4.83s   min=1.09s   med=2.63s    max=18.39s   p(90)=11.92s  p(95)=12.84s 
     http_reqs......................: 1280   2.067612/s
     iteration_duration.............: avg=38.69s  min=15.93s  med=35.97s   max=1m25s    p(90)=48.97s  p(95)=1m25s  
     iterations.....................: 160    0.258451/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m19.1s), 00/10 VUs, 160 complete and 0 interrupted iterations
default ✗ [=====>--------------------------------] 10 VUs  10m19.1s/10m0s  0160/1000 shared iters

Well, a bit later in the day and it is faster now… ha! So, it is varying a bit:

$ date; cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js 
Thu  3 Apr 16:19:19 BST 2025
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
  //http.get('http://vader.lan/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/st-patrick-monument.mp4');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 1.0 GB 1.7 MB/s
     data_sent......................: 416 kB 681 B/s
     dropped_iterations.............: 700    1.145032/s
     http_req_blocked...............: avg=7.08µs  min=1.69µs   med=4.08µs   max=505.02µs p(90)=7.32µs  p(95)=10.04µs
     http_req_connecting............: avg=807ns   min=0s       med=0s       max=402.07µs p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=2.54s   min=952.87ms med=1.72s    max=11.8s    p(90)=6.23s   p(95)=6.57s  
       { expected_response:true }...: avg=2.54s   min=952.87ms med=1.72s    max=11.8s    p(90)=6.23s   p(95)=6.57s  
     http_req_failed................: 0.00%  0 out of 2400
     http_req_receiving.............: avg=1.06ms  min=28.2µs   med=411.04µs max=10.44ms  p(90)=3.15ms  p(95)=4.06ms 
     http_req_sending...............: avg=26.86µs min=5.78µs   med=13.02µs  max=2.81ms   p(90)=29.55µs p(95)=46.39µs
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=2.54s   min=952.81ms med=1.72s    max=11.8s    p(90)=6.23s   p(95)=6.56s  
     http_reqs......................: 2400   3.925825/s
     iteration_duration.............: avg=20.37s  min=11.81s   med=19.47s   max=30.65s   p(90)=25.72s  p(95)=27.58s 
     iterations.....................: 300    0.490728/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m11.3s), 00/10 VUs, 300 complete and 0 interrupted iterations
default ✗ [==========>---------------------------] 10 VUs  10m11.3s/10m0s  0300/1000 shared iters

chriso · April 3, 2025, 3:55pm

2025.1.2.13: 31.34482759%
2025.1.2.12: 6.13793103%
2025.1.2.11: 57.70689655%
2025.1.2.6: 3.58620690%
old: 1.22413793%

Hopefully your numbers will get better as more people get on 2025.1.2.13. We seem to have stalled at ~30% for about 24 hours or so now.

happybeing · April 3, 2025, 4:21pm

31% is way better than the previous release. Last I saw it was under 2%.

I wonder why this is picking up so fast, at least to start.

chriso · April 3, 2025, 4:29pm

The 2025.1.2.12 release didn’t actually contain an update for the node. There was a new binary built with a different package version, but there were no changes. It remained on 0.3.8. Some people must be downloading the binary from the Github release, because you wouldn’t get that binary if you were going via antctl.

Also, I think the numbers we have now are much more accurate, so it’s hard to tell what the state of play really was before the 2025.1.2.13 release.

Traktion · April 7, 2025, 12:23pm

Getting faster!

$ date; cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js 
Mon  7 Apr 08:34:06 BST 2025
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
  //http.get('http://vader.lan/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/st-patrick-monument.mp4');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 1.3 GB 2.2 MB/s
     data_sent......................: 527 kB 869 B/s
     dropped_iterations.............: 620    1.020981/s
     http_req_blocked...............: avg=8.37µs  min=1.76µs   med=4.9µs    max=1.37ms   p(90)=8.26µs  p(95)=11.09µs
     http_req_connecting............: avg=626ns   min=0s       med=0s       max=343.47µs p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=1.99s   min=739.09ms med=1.57s    max=8.11s    p(90)=3.24s   p(95)=6.2s   
       { expected_response:true }...: avg=1.99s   min=739.09ms med=1.57s    max=8.11s    p(90)=3.24s   p(95)=6.2s   
     http_req_failed................: 0.00%  0 out of 3040
     http_req_receiving.............: avg=1.06ms  min=25.84µs  med=448.77µs max=9.36ms   p(90)=3.06ms  p(95)=3.92ms 
     http_req_sending...............: avg=31.14µs min=5.74µs   med=15.36µs  max=6.27ms   p(90)=29.61µs p(95)=44.55µs
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=1.99s   min=738.98ms med=1.57s    max=8.11s    p(90)=3.24s   p(95)=6.2s   
     http_reqs......................: 3040   5.006102/s
     iteration_duration.............: avg=15.98s  min=10.31s   med=15.19s   max=24.03s   p(90)=22.4s   p(95)=23.91s 
     iterations.....................: 380    0.625763/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m07.3s), 00/10 VUs, 380 complete and 0 interrupted iterations
default ✗ [=============>------------------------] 10 VUs  10m07.3s/10m0s  0380/1000 shared iters

Traktion · April 11, 2025, 10:13am

Latest run was similarly quick too:

EDIT: Note that this includes an AntTP change which streams all files, in addition to those with RANGE requests. For localhost, the end result is similar, but for public instances, it reduces the latency (from autonomi → AntTP → client), as chunks are sent (in order) as soon as they arrive from the Autonomi library.

$ date; cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js 
Fri 11 Apr 11:01:11 BST 2025
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 1.3 GB 2.1 MB/s
     data_sent......................: 500 kB 830 B/s
     dropped_iterations.............: 640    1.06312/s
     http_req_blocked...............: avg=8.32µs   min=1.66µs   med=5.3µs    max=776.85µs p(90)=10.48µs  p(95)=13.63µs 
     http_req_connecting............: avg=595ns    min=0s       med=0s       max=270.8µs  p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=2.09s    min=714.56ms med=1.38s    max=15.81s   p(90)=2.98s    p(95)=5.94s   
       { expected_response:true }...: avg=2.09s    min=714.56ms med=1.38s    max=15.81s   p(90)=2.98s    p(95)=5.94s   
     http_req_failed................: 0.00%  0 out of 2880
     http_req_receiving.............: avg=1.65s    min=405.38ms med=968.24ms max=15.52s   p(90)=2.52s    p(95)=5.59s   
     http_req_sending...............: avg=31.92µs  min=6.11µs   med=16.28µs  max=2.1ms    p(90)=37.32µs  p(95)=56.97µs 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=433.09ms min=163.32ms med=416.33ms max=1s       p(90)=610.21ms p(95)=655.58ms
     http_reqs......................: 2880   4.784039/s
     iteration_duration.............: avg=16.72s   min=8.79s    med=14.94s   max=26.8s    p(90)=25.53s   p(95)=26.03s  
     iterations.....................: 360    0.598005/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m02.0s), 00/10 VUs, 360 complete and 0 interrupted iterations
default ✗ [============>-------------------------] 10 VUs  10m02.0s/10m0s  0360/1000 shared iters

Traktion · April 11, 2025, 8:22pm

I tried using 3 medium sized video files to see how that is looking now. It has a much higher throughput and I’m wondering if my wifi may be bottlenecking now. I’ll have to do a run plugged in! ha!

Note that those are megabytes - that’s 220 mbit/s!

$ cat localhost-large-autonomi-http.js; k6 run -u 10 -i 1000 localhost-large-autonomi-http.js
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/91d16e58e9164bccd29a8fd8d25218a61d8253b51c26119791b2633ff4f6b309/to-autonomi.mp4');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/st-patrick-monument.mp4');
  http.get('http://localhost:8080/b6ec9f0f84cf6236dc42d3624679649f51024a57a58b2805552bb3aa690244dd/newcastle-promenade.mp4');
}

/\      Grafana   /‾‾/  
/\  /  \     |\  __   /  /   
/  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
/ __________ \  |_|\_\  \_____/

execution: local
script: localhost-large-autonomi-http.js
output: -

    scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
* default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


data_received..................: 17 GB 28 MB/s
data_sent......................: 60 kB 97 B/s
dropped_iterations.............: 880   1.42613/s
http_req_blocked...............: avg=45.68µs min=2.84µs  med=11.21µs  max=1.64ms   p(90)=35.65µs  p(95)=48.21µs
http_req_connecting............: avg=22.87µs min=0s      med=0s       max=1.35ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=17.13s  min=2.23s   med=11.99s   max=40.66s   p(90)=32.76s   p(95)=38.21s
{ expected_response:true }...: avg=17.13s  min=2.23s   med=11.99s   max=40.66s   p(90)=32.76s   p(95)=38.21s
http_req_failed................: 0.00% 0 out of 360
http_req_receiving.............: avg=15.44s  min=1.15s   med=10.29s   max=40.47s   p(90)=32.54s   p(95)=37.86s
http_req_sending...............: avg=66.51µs min=11.09µs med=38.22µs  max=640.58µs p(90)=127.49µs p(95)=168.42µs
http_req_tls_handshaking.......: avg=0s      min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s
http_req_waiting...............: avg=1.69s   min=10.81ms med=514.21ms max=13.56s   p(90)=5.42s    p(95)=12.93s
http_reqs......................: 360   0.583417/s
iteration_duration.............: avg=51.42s  min=41.84s  med=50.18s   max=1m10s    p(90)=1m1s     p(95)=1m10s
iterations.....................: 120   0.194472/s
vus............................: 10    min=10       max=10
vus_max........................: 10    min=10       max=10


running (10m17.1s), 00/10 VUs, 120 complete and 0 interrupted iterations
default ✗ [===>----------------------------------] 10 VUs  10m17.1s/10m0s  0120/1000 shared iters

tobbetj · April 12, 2025, 12:43am

220 Mbit/s is some serious good numbers for a decentralized network, very acceptable speeds. 25-50 Mbit/s would be very good, anything above that is excellent in my book.

Traktion · April 14, 2025, 10:48pm

On the smaller file test, some further gains today too.

I’ve also made some (unreleased) optimisations to AntTP, which should be helping here too:

$ cat localhost-autonomi-http.js; k6 run -u 10 -i 1000 localhost-autonomi-http.js
import http from 'k6/http';

export default function () {
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_QdxdljdwBwR2QbAVr8scuw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_dH5Ce6neTHIfEkAbmsr1BQ.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_pt48p45dQmR5PBW8np1l8Q.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_sWZ4OWGeQjWs6urcPwR6Yw.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_ZT6qplX5Yt8PMCUqxq1lFQ.png');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_SxkGLnSNsMtu0SDrsWW8Wg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_bogEVpJvgx_gMHQoHMoSLg.jpeg');
  http.get('http://localhost:8080/cec7a9eb2c644b9a5de58bbcdf2e893db9f0b2acd7fc563fc849e19d1f6bd872/1_LFEyRQMHmxRnZtJwMozW5w.jpeg');
}

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: localhost-autonomi-http.js
        output: -

     scenarios: (100.00%) 1 scenario, 10 max VUs, 10m30s max duration (incl. graceful stop):
              * default: 1000 iterations shared among 10 VUs (maxDuration: 10m0s, gracefulStop: 30s)


     data_received..................: 1.9 GB 3.2 MB/s
     data_sent......................: 763 kB 1.3 kB/s
     dropped_iterations.............: 450    0.744449/s
     http_req_blocked...............: avg=7.73µs   min=1.52µs   med=4.49µs   max=1.15ms   p(90)=8.05µs   p(95)=11.46µs
     http_req_connecting............: avg=483ns    min=0s       med=0s       max=560.98µs p(90)=0s       p(95)=0s     
     http_req_duration..............: avg=1.37s    min=637.41ms med=1.11s    max=17.3s    p(90)=1.78s    p(95)=2.24s  
       { expected_response:true }...: avg=1.37s    min=637.41ms med=1.11s    max=17.3s    p(90)=1.78s    p(95)=2.24s  
     http_req_failed................: 0.00%  0 out of 4400
     http_req_receiving.............: avg=890.62ms min=405.67ms med=688.58ms max=17.17s   p(90)=1.12s    p(95)=1.37s  
     http_req_sending...............: avg=27.06µs  min=5.11µs   med=14.85µs  max=1.31ms   p(90)=32.11µs  p(95)=49.59µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=483.03ms min=107.4ms  med=405.61ms max=10.78s   p(90)=723.74ms p(95)=851.5ms
     http_reqs......................: 4400   7.27906/s
     iteration_duration.............: avg=10.99s   min=7.16s    med=9.36s    max=30.22s   p(90)=14.47s   p(95)=20.94s 
     iterations.....................: 550    0.909882/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10


running (10m04.5s), 00/10 VUs, 550 complete and 0 interrupted iterations
default ✗ [===================>------------------] 10 VUs  10m04.5s/10m0s  0550/1000 shared iters

Traktion · April 14, 2025, 11:03pm

From doing some digging, this actually looks like there is some caching within the library helping out here too. Adding more simultaneous clients results in a higher bandwidth to the client, but little difference across my router.

I don’t think it is AntTP doing it, as each request is mostly isolated, but there is a shared library between the connections. I’d like to understand some more though. It only seems to apply when there are lots of connections asking for exactly the same chunk, at about the same time, too.

I’ll do some testing with a variety of concurrent connections to see how that looks too. I suspect the caching is short lived, as the next iteration is regular speed again.

Topic		Replies	Views
Benchmarking Testnets Community	20	2321	September 22, 2017
Memory and performance testing Development	70	1995	September 12, 2022
IntolerantNodeNet [18/09/23 Testnet] [ Offline ] Releases	164	2590	September 30, 2023
Local baby-fleming issues Support	7	669	June 27, 2021
Update 18 January, 2024 Updates	45	1816	January 25, 2024

Performance testing upload/download using AntTP

Related topics