Yes it could be like learning how to use google to get information, or to learn the logic to solve something. I think the main under hyped aspect of AI currently is how useful it can be integrated with whole bunch of regular programming code and UI/UX to implement parts of functionality that can't be programmed explicitly. Like even if gpt4 is the last AI model, if it keeps getting cheaper and faster, the amount of cool features that can be with with some code and a few API calls in parallel or in sequence will add a lot of nice features to all the apps people interact with everyday.
Like my Dad uses multiple apps to get the information he wants for all the sports games that happen during the week, and no app is perfect. If he could prompt something to make a UI with the relevant info he wants displayed like he wants, he would be so much happier. It's little things like that that will make software way more customized and useful for people.
I think there's a lot of hype on AGI and they'll be more, but just as another tool in the programmer's toolbox, LLMs will allow better product to be developed (if people learn how to prompt and think with LLMs in mind vs dismissing all of it when they see a mistake)..