Can application layer improve local model output quality?

Someone pointed me to this post from Cline engineer - below is my response to that

Post: https://cline.bot/blog/why-cline-doesnt-index-your-codebase-...

That post however does not apply to offline processing use case. Here are his 3 main problem points they re trying to solve:

Code Doesn't Think in Chunks

But then he is describing follow semantic links through imports, etc. -> that technique is still hierarchical chunking, and I am planning to implement that as well: it's straightforward.

2. Indexes Decay While Code Evolves

This is just not true - there are multiple ways to solve it. One, for example, is continuous indexing at low priority in the background. Another one - monitoring for file changes and reindexing only differences, etc. I already implemented first iteration for this: index remains current.

3. Security Becomes a Liability (and then goes into embeddings to be stored somewhere)

We are talking about offline mode of operation. Not with Aye Chat: it implements embedding store locally - with ChromaDB and ONNXMiniLM_L6_V2 model.

So as you can see - none of his premises apply here.

And then as part of solution he claims that "context window does not matter because Claude and ChatGPT models are now into 1M context window" - but once again that does not apply to locally hosted models: I am getting 32K context with Qwen 2.5 Coder 7B on my non-optimized setup with 8Gb VRAM.

The main thing why I think it may work is the following: answering a question includes "planning for what to do", and then "doing it". Models are good at "doing it" if they are given all necessary info, so if we unload that "planning" into application itself - I think it may work.