undefined | Better HN

0 pointsdanenania1y ago0 comments

One thing I've found in doing a lot of coding with LLMs is that you're often better off updating the initial prompt and starting fresh rather than asking for fixes.

Having mistakes in context seems to 'contaminate' the results and you keep getting more problems even when you're specifically asking for a fix.

It does make some sense as LLMs are generally known to respond much better to positive examples than negative examples. If an LLM sees the wrong way, it can't help being influenced by it, even if your prompt says very sternly not to do it that way. So you're usually better off re-framing what you want in positive terms.

I actually built an AI coding tool to help enable the workflow of backing up and re-prompting: https://github.com/plandex-ai/plandex

0 comments

withinboredom1y ago

As someone who uses LLMs on my hobby projects to write code, I’ve found the opposite. I usually fix the code, then send it in saying it is a refactor to clarify things. It seems to work well enough. If it is rather complex, I will paste the broken code into another conversation and ask it to refactor/explain what is going on.

danenaniaOP1y ago

Fixing the mistake yourself and then sending the code back is a positive example, since you're demonstrating the correct way rather than asking for a fix.

But in my experience, if you continue iterating from that point, there's still a risk that parts of the original broken code can leak back into the output again later on since the broken code is still in context.

Ymmv of course and it definitely depends a lot on the complexity of what you're doing.

gopher_space1y ago

I’m attempting to keep the context ball rolling by reiterating key points of a request throughout the conversation.

The challenge is writing in a tone that will gently move the conversation rather than refocus it. I can’t just inject “remember point n+1” and hope that’s not all it’ll talk about in the next frame.

If nothing else, LLMs have helped me understand exactly why GIGO is a fundamental law.

j / k navigate · click thread line to collapse