> how do you know that the code it just created actually does what you want it to
I believe that problem can be solved, and will be with the correct new tools.
I think a new type of "low-code" programming language will evolve tailored for domain experts that don't know coding, but understand how to guide an LLM.
The development environment should be a crossover between a project-management tool (Jira or Azure devops style) with literate programming: something that allows organising all the separate parts that compose the program in a way understandable by the human and the AI bot.
Online notebooks are a good first step in that direction, in special the "reactive" kind where the runtime logic does not depend on the order you change things in the page.