Adding verification steps to AI agents made them worse. I tested it 29 times (opens in new tab)

(substack.com)

3 pointschrisdudek1mo ago4 comments

4 comments

> What works... External mechanical checks

This is what I expected verification to mean going in.

For some reason verification in this article means "try to convince the AI even harder than before"

I first tried to make the agent follow concrete paths, but no tricks worked. The reason I wanted to achieve it this way is that I wanted to create universal "rules" that would be followed without additional CLI or hooks. It didn't work out as expected. I needed it for my project, and currently I started to enforce more things with CLI, which is directly accessible by an agent, so it's now "guided" more forcefully.

j / k navigate · click thread line to collapse

4 comments

Bjartr1mo ago

> What works... External mechanical checks

This is what I expected verification to mean going in.

For some reason verification in this article means "try to convince the AI even harder than before"

chrisdudekOP1mo ago

j / k navigate · click thread line to collapse