Ceph setup has a few magic moments, but after you accept and learn those incantation, it just works! Highly recommend. Used to be easy to get started with their all-in-one container image.

Ceph is demanding to learn, but once you do It Just Works very reliably. The cephadm tool makes it quite easy to bootstrap and manage a vanilla cluster, I'd highly recommend bootstrapping a toy cluster with it if you're interested in learning. Everything can run in a container, and there's a decent webui that's useful for learning and visualization.