We can revisit the graveyard of over-hyped technologies if you want, and catalog the failures and wasted money. Ride the wave if you like. Show the success stories of LLMs in terms of business value or innovation. Don’t just repeat the press releases and anecdotes. You won’t find many successes, though maybe they will come in time.
> I’ve found [an LLM] good at writing SQL queries.
I don’t mean snark, but how do you know it writes good queries? I will grant that it can write queries that execute, but do those give correct results? Will they look good on a large production database where performance starts to matter?
When I played with LLMs writing SQL they worked well enough on toy schemas that resemble those found in tutorials. But faced with a real schema that requires joining tables and applying complex conditions they failed to produce usable SQL. However they did (with prodding) produce SQL that executed. It just gave the wrong results. That’s not a gain along any axis and could cost a lot of money for an organization blindly trusting such tools.
LLMs can write “good” code for some constrained definition of “good.” If you also want correct, testable, maintainable, secure, etc. they don’t perform well enough. Not just my opinion, we’ve had time to study the output of coding LLMs and it’s not looking good.
