Without some robust general intelligence in the loop, whether that be AGI or human, computers will always have countless strange, unpredictable, and devastating failure modes, especially if given increasing responsibility.

This is the reason why few systems of significant complexity and risk exist today without humans in the loop.