I hate to be blunt, but the moral of this story is that you should not trade good operational practice for some personal preferences. Distros generally don't matter, until they do. Choices made at the distro level, like, hey lets build world! don't mesh well into providing stable and reliable service.
You can run any distro you want, safely, with best practices. Which means you aim to not make upstream changes which have a high probability of breaking things ... like glibc ... and you focus upon maintaining a stable platform for your service. You can do that with any distro. Any unix(-like) thing really. There's no magic, just common sense.
The flip side is that some practices encouraged by various distros range between glacial change (read as stability), or near relativistic change (read as here be dragons). Gentoo encourages the latter, and Debian/CentOS encourage the former. This does not mean one is better than the other, just that you have to pay more attention with some distros, to maintaining operational discipline.
I would have run the Ceph in a VM, with the disks passed through so I could downgrade/crossgrade to different Ceph versions while having a rollback plan that is the identity function.
You can run any distro you want, safely, with best practices. Which means you aim to not make upstream changes which have a high probability of breaking things ... like glibc ... and you focus upon maintaining a stable platform for your service. You can do that with any distro. Any unix(-like) thing really. There's no magic, just common sense.
The flip side is that some practices encouraged by various distros range between glacial change (read as stability), or near relativistic change (read as here be dragons). Gentoo encourages the latter, and Debian/CentOS encourage the former. This does not mean one is better than the other, just that you have to pay more attention with some distros, to maintaining operational discipline.