[MemDebugNet] [4/10/23 Testnet] [Offline]

Development Releases
21

In the second test (same file) a similar error with a different chunk.

0: Transfer Error Failed to send tokens due to Network Error Could not retrieve the record after storing it: 5d6b17a589bee1bec473ab316a129c80bc55429665bd1e7db7bc48bda755ad8b…

logs.zip (312.7 KB)

3 Likes
22

trying to run some heaptrack nodes so I did my own build with just cargo build --release no luck though, .zst’s are empty. how goes yours?

Edit, as you mentioned before once I killed the process it was written to, but the nodes were getting no chunks unlike all the rest without heaptrack which are doing just fine. whats up with that.

2 Likes
23

not going so good here was trying to upload an image of Ubuntu 4.6Gb

safe files upload ubuntu-23.04-desktop-amd64.iso

failed with

Transfers applied locally
Error:
   0: Transfer Error Failed to send tokens due to Network Error Could not retrieve the record after storing it: e3b72da3d9bfaaa5f8313e322807e5281f8f52ffc95333fe6a957c31b62ecfa1..
   1: Failed to send tokens due to Network Error Could not retrieve the record after storing it: e3b72da3d9bfaaa5f8313e322807e5281f8f52ffc95333fe6a957c31b62ecfa1.

Location:
   sn_cli/src/subcommands/files.rs:209

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Logs

safe.log 64051842633bba0c45375911e1fa722bc113df7238b46cc916a91442f3618baa
safe.log.20231004T124058 80bcafb36fdf2064e0abd738be97f90d369e935f4f7018cd085631ae78a00a2d
1 Like
24

also was trying to upload a directory of 40Gb mp3’s on a different machine

safe files upload ./

failed with

Transfers applied locally
Error:
   0: Transfer Error Failed to send tokens due to Network Error Could not retrieve the record after storing it: b2b42d7e49d61ac252fe908a274556f50c76430f278ee6947f186570457e4d43..
   1: Failed to send tokens due to Network Error Could not retrieve the record after storing it: b2b42d7e49d61ac252fe908a274556f50c76430f278ee6947f186570457e4d43.

Location:
   sn_cli/src/subcommands/files.rs:209

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Log

safe.log to d3c03d45b3bebe0d1002a7d7a5e859192bc08d8da2cb9e482dfc7c04e7f7a4ef
1 Like
25

We have some data. So I’d say dont sweat it too much. (Your tracks may be zero until you kill the process of heaptrack process btw. I still didn’t work out why/when it’s written).

Oh interesting. I don’t know. Will double check the status of ours now.

thanks @digipl / @aatonnomicc I’ll have a look now.

4 Likes
26

This has never happened to me before (snert). Haven’t been able to get any tokens with safe wallet get-faucet 138.68.161.29:8000. I can connect to the Network but it just dumps out without error. Tried both Client versions 0.83.17 and 0.83.18. Deleted everything and started fresh. Still no luck. Windows 10. Log attached.

log_2023-10-04_08-51-43.zip (4.7 KB)

4 Likes
27

Failed to read cash_note file again here.

Can you quickly double check if this exists: /Users/<your user name>/Library/Application Support/safe/client/wallet/created_cash_notes/2518f70e91ca18129036d42973dcaec8bcaf532541de52b79a0b4608a650dc8c.cash_note @digipl

3 Likes
28

Ah, i cannot faucet either. Interesting. May have died :thinking:

3 Likes
29

same here faucet is not to faucetable at the moment

1 Like
30

Exist…
2518f70e91ca18129036d42973dcaec8bcaf532541de52b79a0b4608a650dc8c.cash_note.zip (29.7 KB)

2 Likes
31

Okay great, that means it’s a concurrency file read/write issue (and the client just not retrying after that… which i’d thought we’d sorted but apparently not!)

edit: faucet is back up now. we’re looking into the fail there.

8 Likes
32

you have the same issue as digipl here, we must have enabled erroring out in a new place and that’s tripping us up with larger updates/more concurrency.

4 Likes
33

This is the wallet get-faucet cmd that’s erroring out? (or not as the case may be?)

2 Likes
34

It’s all good now since the faucet went back up. I’m now able to get tokens.

6 Likes
35

Something else is going on with me.

@Southside if I can figure it out after doing husband duties hopefully it sheds light on why your nodes at home get killed.

I added a 4th machine to the mix.
Exact same setup as the previous 3 I have been running no issues with several tests being behind NAT.

The new machine with the exact same port forwarding rule keeps getting its noded killed.
And for good measure I added some arm nodes too which are also getting nodes booted.

Anyway need to go trim bamboo before I am allowed to investigate.

@storage_guy tracking connections now. Not yet pushed to github, will see how it all pans out first.

3 Likes
36

FYI: our heaptracked nodes are getting records as faras i can see, just FYI.

If heaptracking is problematic we should have enough data (so far, anything approaching a leak is just connection management from libp2p, so I think nothing interesting there yet)

4 Likes
37

Just stay up all night.

81exzz

9 Likes
38

Is this a euphemism? #fnaar

2 Likes
39

I an getting the familiar deserialized bytes don't encode a group element error, even just trying to get some tokens

"/ip4/68.183.33.142/tcp/34385/p2p/12D3KooW9qo2TEV6jTrFe8JCTs4CdGwjoZHKVFwpiWmjyQGRxN9a", "/ip4/209.97.143.12/tcp/45085/p2p/12D3KooWCKZyxndJoL7snv1w6q4WkgsDzQB6n3rPFVCCSpcePbVd", "/ip4/161.35.161.196/tcp/44469/p2p/12D3KooW9uZJEnTm2eg6CrMvbepn5RSgkAMU4RQsNzk9nmBbwX55"]...
🔗 Connected to the Network                                                                                            Error: 
   0: Bincode error:: deserialized bytes don't encode a group element
   1: deserialized bytes don't encode a group element

Location:
   sn_cli/src/subcommands/wallet.rs:176
1 Like
40

no luck with windows today. error from faucet and still empty in folder record_store after running a node for 20 plus minutes

image
image1915×567 57.3 KB

4 Likes