To add on to this, I see many complaints that "[AI] produced garbage code that doesn't solve the problem" yet I have never seen someone say "I set up a verification system where code that passes the tests and criteria and code that does not is identified as follows" and then say the same thing after.
To me it reads like saying "I typed pseudocode into a JS file and it didn't compile , JS is junk". If people learn to use the tool, it works.
Anecdotally, I've been experimenting with migrations between languages and found LLMs taking shortcuts, but when I added a step to convert the source code's language to an AST and the transformed code to another AST and then designed a diff algorithm to compare the logic is equivalent in the converted code, and to retry until it matches within X tolerance, then it stopped outputting shortcuts because it simply would just continue until there were no shortcuts made. I suspect complainants are not doing this.
