Storage proceeding

[neo]

@Seneca

it’s just a few extra data chunks that farmers get paid for hosting. And farmers only look likely to get screwed in your own head due to the assumptions you make.

Used that quote to revive the discussion on storing data correction.

If the reference (see below) is still valid then when the chunk is being transmitted to the vault there is a measure of this in place, but not stored, just to ensure chunk reaches the vault quickly and intact. This is not storing extra error correcting info in the vault, just each chunk transmission error detection/correction.

@eblanshey

A minimum of 28 out of 32 of these client manager nodes will then pass their chunk pieces to groups of 32 data managers whose IDs most closely match the chunk IDs (which is why the chunk ID also acts as its location, as mentioned above), using xor networking (described later). In this way the transfer of info can withstand small loss (up to 4 pieces) without retransmitting the whole data again (this is used in many places). This process is called the scatter <-> gather approach and uses Rabin’s Information Dispersal Algorithm. The data managers’ job is to distribute the chunk they received to nodes on the network (with vault ranks always being under consideration), and to continuously make sure there are always at least 4 copies available. At this point, each broken up chunk is now in its own data manager group. The data managers recompile the pieces into whole chunks at this point.