> But on the nuclear issue, it's not a good sign that he's explicitly saying that this AGI future is a threat to nuclear deterrence and the triad. Like, where do you go up from there? That's the highest level of alarm that any government can have. This isn't a boy crying wolf, it's the loudest klaxon you can possibly make.

This is not new. Anthropic has raised these concerns in their system cards for previous versions of Opus/Sonnet. Maybe in slightly more dryer terms, and buried in a 100+ page PDF, but they have raised the risk of either

a) a small group of bad actors w/ access to frontier models, technical know-how (both 'llm/ai how to bypass restrictions' and making and sourcing weapons) to turn that into dirty bombs / small nuclear devices and where to deploy them. b) the bigger, more scifi threat, of a fleet of agents going rogue, maybe on orders of a nation state, to do the same

I think option a is much more frightening and likely. option b makes for better scifi thrillers, and still could happen in 5-30ish(??) years.

I agree that it is not a good sign, but I think what is a worse sign is that CEOs and American leaders are not recognizing the biggest deterrent to nuclear engagement and war in general, which is globalism and economic interdependence. And hoarding AI like a weapons stockpile is not going to help.