It's either that or it's the monster that is Kubernetes.
And then there's also Nomad, which is drastically simpler than k8s. Not Heroku-easier, but closer to docker-compose than Kubernetes.
Self-plugging my article on the Nomad vs Kubernetes subject: https://atodorov.me/2021/02/27/why-you-should-take-a-look-at...
So now you're stuck with 3 consul clusters, a vault cluster, whatever you choose for config management, and a nomad cluster. Feels like you didn't gain much from the simplicity of nomad.
In addition to that, knowing Hashicorp's pricing, I bet that set up would run you north of a million a year for enterprise setup.
Nomad relies on Consul for Service Discovery and K/V storage, and Vault for secrets, indeed ( Vault can use a variety of backends, including an integrated Raft-based one, Consul, object storage, etc.). One tool that does one thing well, which integrates with other tools that their thing well.
I vastly prefer having three simple Raft-based clusters to manage than the "everything and the kitchen sink" approach Kubernetes takes, with results like base64 encoded for "secrets".
And as someone doing both, Nomad+Consul+Vault are drastically easier on day one and day two. They're also usable outside of Nomad ( you can have bare metal machines outside of Nomad using Vault secrets and Consul for SD and K/V), and you can link multiple regions together.
You do indeed need some basic config management to configure the clusters. Ansible seems to have won that race sadly, and there are available playbooks.
> In addition to that, knowing Hashicorp's pricing, I bet that set up would run you north of a million a year for enterprise setup
They've changed up their pricing structure and there are more tiers and add-ons, so i doubt it ( basic Vault cluster was in the 5 figured range per year), if you need support. It's not like enterprise support for Kubernetes would come cheap either, especially if you do it the recommended way with multiple clusters and all that.
That's normal, since Nomad is only about deployment, where k8s is about full cluster management.
I'm sure there's still a gap to be bridged there between that and a PaaS which you literally just add as a git remote. But I don't think it's huge.
Habing been on both sides of the isle, in my opinion, K8s has great ux for consumers, but for is a nightmare for ops teams who maintain it. For a self-hosted version anyway.
Now, all that said, Canonical certainly advertises microk8s as being production-ready, production-grade, and suitable for use in production environments, for example in [1]. It definitely seems like it's meant to be far more serious than, say, minikube, which explicitly is just for local development.
Can you speak to specific limitations with microk8s, or point to resources which go into more depth on this?
