There are two primary issues to solve:

1: Protecting against bad things (prompt injections, overeager agents, etc)

2: Containing the blast radius (preventing agents from even reaching sensitive things)

The companies building the agents make a best-effort attempt against #1 (guardrails, permissions, etc), and nothing against #2. It's why I use https://github.com/kstenerud/yoloai for everything now.