I think the right solution is to endow the LLM with just enough permissions to do whatever it was meant to do in the first place.

In the customer service case, it has read access to the customer data who is calling, read access to support docs, write access to creating a ticket, and maybe write access to that customer's account within reason. Nothing else. It cannot search the internet, it cannot run a shell, nothing else whatsoever.

You treat it like you would an entry level person who just started - there is no reason to give the new hire the capability to SMS the entire customer base.