Tiered Storage in Farming Rig

One of the big reasons I did the port of the MaidSafe codebase to FreeBSD was to avail of ZFS. Our storage layer is actually currently very, very simple, it literally stores chunks as files by opening a new file and blatting the chunk into it, similarly to read chunks we currently open a handle to it and blat the chunk into memory. We don’t, currently, even use memory mapped i/o, we just use the standard iostreams file i/o built into the C++ standard library.

As a result, with the present extremely naive storage layer there is no concept of storage tiering, or anything fancy at all. You basically need a local not distributed filing system, and that’s it. As a result, performance is heavily influenced by what a single non-tiered filing system can deliver. And of all the local not distributed filing systems presently available, the clear performance leader is ZFS which can provide truly astonishing synchronous write performance (i.e. write a chunk and fsync it, this murders most filing systems) plus it makes very good use of any RAM you feed it as a read cache, it’s far far far better at this than ext4 or NTFS, and it self heals bitrot which means the SAFE network won’t penalise your node for serving corrupted chunks.

Hence it is my guess that for those looking into big iron farming, the sole choice of OS will be FreeBSD and the sole choice of storage will be ZFS paired with a fast SSD write intent log device. Hence me doing the port to FreeBSD 10 where we now regularly CI test to ensure we don’t break things.

Niall

6 Likes