Yeah, it's an arms race, and OpenAI does stand to lose. But this is a prisoner's dilemma situation. OpenAI can shut themselves down, but that doesn't fix the problem of someone creating a dangerous situation, as everyone else will keep going.
The only way to actually stop it is to get everyone to stop at once, via regulation. Otherwise, stopping by yourself is just a unilaterally bad move.
That's the charitable explanation, at least. These days I don't trust anything Musk says at face value, but I do think that AI is driving society off a cliff and we need to find a way to pump the breaks.
