Then I remembered the times I've worked at large companies and depended on code written by other teams. I didn't review every line of code they had written - I'd trust that they had done a competent job, integrate with that code myself, and only dig into the details of their code if I run into bugs or performance issues or other smells that something was wrong.
Trusting humans is obviously different from trusting AI - humans have reputations, and social contracts, and actual intelligence as opposed to multiplying matrices and rolling a dice. But... I do think an AI model can still earn trust over time. I've spent enough time with Opus 4.5 and 4.6 that I trust them not to make dumb mistakes with the common categories of code that I use them for. Of course now I need to rebuild that trust with 4.7!
I think the most interesting challenge here is to figure out how to have coding agents demonstrate that the code works without actually reading every line of it yourself - in the same way that I might ask an engineering team I haven't worked with before for a demo and then interrogate them about their testing strategy before relying on their work.
If the engineering team fucks up somehow they can be kept accountable. An AI cannot be held accountable.
As an engineering manager I can take accountability for the output of my team even if I don't review every line. Using coding agents feels similar.
When I delegate to the AI there’s no pressure to deliver good work. There’s no performance review, threat of firing, or performance bonuses.
All the responsibility and accountability flows upwards on the person orchestrating, but there is not hierarchy of responsibility like there is with people.
A computer can never be held accountable
Therefore a computer must never make a management decision
https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
The SaaS companies disrupting today could become utilities offering mechanized leases tomorrow.
With agents as a singular "swarm brain" (per machine, not a global hivemind) just seems like a natural course of abstraction.
