Chat completion sends the full prompt history on every call.
I am working on my own coding agent and seeing massive improvements by rewriting history using either a smaller model or a freestanding call to the main one.
It really mitigates context poisoning.
Which as far as I understand it is summarizing the context with a smaller model.
Am I misunderstanding you, as the practical experience of most people seem to contradict your results.
Once we start to see that kind of self feedback going in next iterations (w/ possible training runs between sessions, "dreaming" stage from og RL, distilling a session, grabbing key insights, storing them, surfacing them at next inference, etc) then we'll see true progress in this space.
The problem is that a lot of people work on these things in silos. The industry is much more geared towards quick returns now, having to show something now, rather than building strong fo0undations based on real data. Kind of an analogy to early linux dev. We need our own Linus, it would seem :)
You can also ask the llm at the end of a feature chat to prepare a prompt to start the next feature chat so it can determine what knowledge is important to communicate to the next feature chat.
Summarizing a chat also helps getting rid of wrong info, as you’ll often trial and error towards the right solution. You don’t want these incorrect approaches to leak into the context of the next feature chat, maybe just add the “don’t dos” into a guidelines and rules document so it will avoid it in the future.
How do you define success of a model's run?
