You and I are advocating for the same exact solution here! Don't give your LLM over-privileged access to production systems.

My way of explaining that to people is to say that it's dangerous to do things like that.

> Junior Engineers aren't "dangerous" but we just don't give them too much access/authority too soon.

If it is not dangerous to give them this access, why not grant it?

I thought the general point is that you can't "force" an LLM to stay within certain boundaries without placing it in an environment where it literally has no other choice.

(Having said that, I'm just a kibitzer.)

I was diagnosing an issue in production. The idea was to have the LLM would need to collect the logs of a bunch of pods, compare the YAML code in the cluster with the templates we were feeding ArgoCD, then check why the original YAML we were feeding the cluster wasn't giving the results we expected (after several layers of templating between ArgoCD Appsets, ArgoCD Applications, Helm Charts and Prometheus Operator).

I have a cursor rule stating it should never make changes to clusters, and I have explicitly told it not to do anything behind my back.

I don't know what happened in the meantime, maybe it blew up its own context and "forgot" the basic rules, but when I got back it was running `kubectl patch` to try some changes and see if it works. Basically what a human - with the proper knowledge - would do.

Thing is: it worked. The MF found the templating issue that was breaking my Alertmanager by patching and comparing the logs. All by itself, however by going over an explicit rule I had given it a couple times.

So to summarize: it's useful as hell, but it's also dangerous as hell.

Exactly. I'll need to dig deeper into its allowlist and try a few things.

Problem is: I also force it to run `kubectl --context somecontext`, as to avoid it using `kubectl config use-context` and pull a hug on me (if it switches the context and I miss it, I might then run commands against the wrong cluster by mistake). I have 60+ clusters so that's a major problem.

Then I'd need a way to allowlist `kubectl get --context`, `kubectl logs --context` and so on. A bit more painful, but hopefully a lot safer.