Terraform's provider model is fundamentally broken. You cannot spin up a k8s server and then subsequently use the k8s modules to configure the server in the same workspace. You need a different workspace to import the outputs. The net result was we had like 5 workspaces which really should have been one or two.
A seemingly inconsequential change in one of the predecessor workspaces could absolutely wreck the later resources in the latter workspaces.
It's very easy in such a scenario to trigger a delete and replace, and for larger changes, you have to inspect the plan very, very carefully. The other pain point was I found most of my colleagues going "IDK, this is what worked in non-prod" whilst plans were actively destroying and recreating things, as long as the plan looked like it would execute and create whatever little thing they were working on, the downstream consequences didn't matter (I realize this is not a shortcoming of the tool itself).
It’s fair to complain that terraform requires weird areas of expertise that aren’t that intuitive and take a little bit of a learning curve, but it’s not really fair to complain that it should prevent bad practices and inexperience from causing the issues they typically do.
https://registry.terraform.io/providers/hashicorp/kubernetes...
> The most reliable way to configure the Kubernetes provider is to ensure that the cluster itself and the Kubernetes provider resources can be managed with separate apply operations. Data-sources can be used to convey values between the two stages as needed.
There are shortcomings in the kubernetes provider as well that make wanting to maintain that in one state file a nonstarter for me.
That's what I expected lifecycle.prevent_destroy to do when I first saw it, but indeed it does not.
That said the failure mode is also a bit more than "a badly reviewed PR". It's:
* reviewing and approving a PR that is removing a resource * approving a run that explicitly states how many resources are going to be destroyed, and lists them * (or having your runs auto approve)
I've long theorised the actual problem here is that in 99% of cases everything is fine, and so people develop a form of review fatigue and muscle memory for approving things without actually reviewing them critically.
