In my experience, it doesn’t matter how good or detailed the prompt is—after enough lines of code, the LLM starts making design decisions for you.
This is why I don’t accept LLM completions for anything that isn’t short enough to quickly verify that it is implemented exactly as I would have myself. Usually, that’s boilerplate code.
^ This. This is where I've landed as far as the extent of LLM coding assistants for me.
They don't
> Garbage in = garbage out generally.
Generally, this statement is false
> When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well.
Keyword: can.
They can also not perform really well despite all the management and materials.
They can also work really well with loosey-goosey approach.
The reason is that they are non-deterministic systems whose performance is affected more by compute availability than by your unscientific random attempts at reverse engineering their behavior https://dmitriid.com/prompting-llms-is-not-engineering
People are expecting perfection from bad spec
Isn’t that what engineers are (rightfully) always complaining about to BD?
I've definitely also found that the poor code can sometimes be a nice starting place. One thing I think it does for me is make me fix it up until it's actually good, instead of write the first thing that comes to mind and declare it good enough (after all my poorly written first draft is of course perfect). In contrast to the usual view of AI assisted coding, I think this style of programming for tedious tasks makes me "less productive" (I take longer) but produces better code.
Not really, not always. To anyone who’s used the latest LLMs extensively, it’s clear that this is not something you can reliably assume even with the constraints you mentioned.
No they don't, they generate a statistically plausible text response given a sequence of tokens.
