Maybe, if we're lucky, Kent Overstreet (koverstreet[3]) will magically appear and talk more on his latest creation.
[1] https://bcachefs.org/bcachefs-principles-of-operation.pdf
[2] https://www.patreon.com/bcachefs (de facto bcachefs development blog)
- Ceph with data loss protection
- Any filesystem on linux-md without data loss protection, BTRFS on linux-md on synology
"The block pointer can store up to three copies of the data each pointed by a unique DVA. These blocks are referred to as “ditto” blocks in ZFS terminology." [2]
And although VMWare calls RAID5/6 "Erasure Coding" [3](!!), I'd say it's not [4].
0: https://docs.oracle.com/en/operating-systems/solaris/oracle-solaris/11.4/manage-zfs/copies-property.html
1: https://jrs-s.net/2016/05/02/zfs-copies-equals-n/
2: https://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf
3: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.virtualsan.doc/GUID-AD408FA8-5898-4541-9F82-FE72E6CD6227.html
4: https://www.itprotoday.com/storage/erasure-coding-vs-raid-which-right-and-when
Edit: some extra info and links
I wonder if Brtfs is using this for raid. Though last I read it Raid-5 was still broken.
For example, Linux MD RAID 5 uses XOR, and RAID 6 uses Galois fields. [1][2]
0: https://www.itprotoday.com/storage/erasure-coding-vs-raid-which-right-and-when
1: https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm#Raid_5
2: http://igoro.com/archive/how-raid-6-dual-parity-calculation-works/The conclusion:
"Analysis and Conclusion
With this test we were looking to confirm that Erasure Coding on commodity hardware can be every bit as fast as dedicated hardware - without the cost or lock-in. We are happy to confirm that even running at top-of-class NIC speeds we will only use a minor fraction of CPU resources for erasure coding on all of the most popular platforms.
This means that the CPU can spend its resources on handling IO and other parts of the requests, and we can reasonably expect that any handling of external stream processors would take at least an equivalent amount of resources.
We are happy to see that Intel improved throughput on their latest platform. We look forward to testing the most recent AMD platform, and we expect its AVX512 and GFNI support to provide a further performance boost. Even if Graviton 3 turned out to be a bit behind, we don’t realistically see it becoming a significant bottleneck. For more detailed information about installing, running, and using MinIO in any environment, please refer to our documentation. To learn more about MinIO or get involved in our community, please visit us at min.io or join our public slack channel."
It's missing a benchmark of the latest AMD CPUs (Genoa/Epyc 4).
Back then CPUs had less cores, RS coding was relatively more expensive and certainly CPU bound on then new NVMe flash devices
It's possible, even likely this is the result of that
GFNI Was only added to AMD CPUs in Zen4, basically the "just released" server CPUs (codename: Genoa) which are currently quite difficult to procure, even 6 months after announcement.