You can't hold the computer accountable, but you can still hold the companies that deployed the code responsible, and if they do a poor job, the business will suffer. And why can't an AI also make the product decisions? It's already writing the code.
I think the "billion dollar single employee plus AI company" is still a ways away, but not actually that far.