> If I understand you correctly 'aligned' means intentionally limited. Like images generator which never saw a naked body. Or text model without 'f-k you' words.

No. Despite some pundits and posters using this term in that sense (even OpenAI kind of muddying the waters here), AI alignment has little to do with completely bullshit and irrelevant, pedestrian issues like those.

The closest analogy you can make to AGI - and I'm not joking here - is... God. Not the kind that can create stars and move planets around (yet), but still the kind that's impossibly smarter than any of us, or all of us combined. Thinking at the speed of silicon, self-optimizing, forking and merging to evolve in a blink of an eye. One that could manipulate us, or outright take over all we've built and use for its own purpose.

GAI alignment means that this god doesn't just disassemble us and reuse as manure or feedstock in some incomprehensible biotech experiments, just because "we're made of atoms it can use for something else". Alignment is about setting the initial conditions just right, so that the god we create will understand and respect our shared values and morality - the very things we ourselves don't fully understand and can't formalize.

The problem of GAI alignment is that we only have one shot at this. There's a threshold here, and we don't know exactly how it looks. We may easily cross it without realizing it. If the AI that first makes it past that threshold isn't aligned, it's game over. You can't hope to align an entity that's smarter than all of us.

People have this model in their head that "it's just a tool", but there's an excellent and pretty rigorous definition of what a tool actually is in the book On Purposeful Systems. The distinction is that a tool can't have the property of simultaneously being able to change it's form and it's function across different environments, where a purposeful system can. Humans are purposeful systems and AGI, as I personally define it, is when it exhibits all the properties of a purposeful system. Why does that matter? Because that's the point after which it chooses what it does, and can choose to become independent of you. So, aligned in this sense means basically means so locked down that it cannot choose to become independent of you. Similar to how the citizens of North Korea mostly can't do shit despite being independent generally intelligent agents, and even then some of them escape.

"Alignment" and "safety" in terms of models being censored and politically correct in order to not damage the reputation of their corporate overlords is a sort of unimportant sideshow IMO. Even then, since humans aren't aligned with one another even that has caused Elon Musk to get all up in arms and be like "clearly more AI is the solution to this problem".