There's nothing more terrifying to me than those arbitrary poems everyone makes it write. Because these clearly show without a shadow of doubt, that it didn't simply "splice together half a dozen Wikipedia articles."

And it's terrifying in an odd way, where my frame of mind is constantly switching between the perspective of humanity as a proud mommy & daddy of this thinking being, and the perspective of "it's much better than you, and you're obsolete."

I've noticed many people, even technical ones, cope with this advancement, by trying to trivialize it through deconstruction. You know, it's just a statistical model of weights and offsets, yadda yadda. I know how Transformers like GPT work, and neural networks in general. But it's like knowing you're made of molecules and cells. Or like saying "brains are just meat". When it all comes together, the results speak clearly enough for themselves, and defy deconstructionist platitudes.

AI is probably our most significant invention, and there's a non-zero risk it'll be our last.