Is the metrics crate under--reporting the actual number of records stored?

Development
1

As the title says , I have my suspicions.

There is a wee script up now at
GitHub - safenetforum-community/chunk_checker: Wee script to see if metrics is honestly reporting actual number of records stored

and I have raised Possible under-counting of records stored · Issue #2775 · maidsafe/autonomi · GitHub

Please consider running this quick script to see if you are in fact storing considerably more records than is being reported.

Paging @chriso @joshuef

This script assumes that

  • each instance of a dir named record_store equates to one node you are running ← it makes no judgement on the health of that node
  • each file in a dir named record_store equates to one record(chunk) stored

chunk_checker.sh script is now available at 3ffa427480548a98ef71ea7d7f937dfc730db1598e0928027a47475798a08748

Uploaded with ant 0.3.8-RC.1

Successfully encrypted file: "chunk_checker.sh"
Paying for 4 chunks..
Uploading file: chunk_checker.sh (4 chunks)..
Successfully uploaded chunk_checker.sh (4 chunks) to: 5ca39839f6c31f10a560d3f7fc7ac8c8f1c62809e0fe6b9507eb2d8adb1878c3
Upload of 1 files completed in 301.04369335s
Uploading public archive referencing 1 files
Successfully uploaded: chunk_checker.sh
At address: 3ffa427480548a98ef71ea7d7f937dfc730db1598e0928027a47475798a08748
Number of chunks uploaded: 8
Number of chunks already paid/uploaded: 0
Total cost: 78247180488 AttoTokens
1 Like
2

I can confirm that I have some nodes with one entry in the record_store. Most don’t have any as well.

But the ones with one record stored do not show up in my monitoring script that reads the /metrics. The /metrics show zero for both the record stored entry and the responsible records field

1 Like
3

So what do we believe?

What the metrics crate says or what we see in the filesystem?

Anyway we will see if anybody responds to Possible under-counting of records stored · Issue #2775 · maidsafe/autonomi · GitHub

4

It might be as you thought, its one out. I didn’t notice in earlier networks because there were always plenty of records.

Or it could be because as someone else pointed out, that its a record stored but no longer considered close enough to keep track of. I seem to remember that the devs said records are only deleted from disk when record space is needed. Old unused records only cleared out when record count is over a certain value.

2 Likes
5

Nodes are too empty for that to happen right now.
Need to get others to run that script to see for themselves but I am now convinced metrics is only counting from the second chunk in any record_store dir.

1 Like
6

You have been recognised for bringing attention to this bug. We will get more accurate reporting soon.

1 Like