aka techbros
The combination of endless trend-chasing, software churn, and techbro culture made me hate everything about software, so I jumped ship to biology.
I think part of your naiveté was thinking this goal was likely to turn out as a net benefit for humanity. Maybe it eventually will. But the current scenario was always the most likely scenario for machines rivaling or surpassing humans in intelligence.
It's being the tech lead of a team of junior to mid level developers. You design roughly what the solution should look like, split it into reasonable sized tasks so they don't go off the deep end, advise them on some of the details, then assign them the tasks and let them get on with it, keeping an eye on what they're doing, reviewing their output, and course correcting them when they go wrong.
Just like with a team of humans, you have to use your judgement as to how much supervision they need individually and how large a task you can give them without them going off the rails.
Not everyone wants to be a team lead not doing coding any longer.
The models get better and better at understanding the intent of a prompt and doing more useful work with less intervention.