Maybe a stupid question but aren't the Amish essentially this thought process put into practice?

They aren't able to outcompete their neighbouring societies economically yet are in no way impoverished, and in many cases actually come out on top by many QOL/health metrics.

Yeah, it’s the same problem as nuclear proliferation.

We see what happened with Ukraine when they gave up their nukes in exchange for a promise of protection.

The same could be true of countries who forgo AI development directly in exchange for promises.

this is Amodei's position in a nutshell for AI development. We have to go as fast as possible because China. It's not the only frame though. If AI models and warfare (cyber, biological) becomes easily accessible and dangerous enough, there is a strong incentive for the world's leaders to cooperate towards something akin to nuclear non-proliferation.

In fact, there's strong incentives now to slow down AI progress for multiple reasons: de-escalate tension over Taiwan and lessen China's desire to build their own advanced fabs, protect peoples livelihoods by smoothing the AI transition. Except the incentives to bring AI companies public (and maintain some twisted shred of American Hegemony) are greater.

Unless you change the balance of selection pressures…

"Darwinian logic of reality"

What does outcompete economically mean and why would it matter? Or do you mean society A dominates in some form society B? This has already happened in history and is the essence of capitalism. If you want to overcome this situation you need to replace capitalism globally.