The Idea in a nutshell:
"Hallucinations" aren't indicative of bad training, but per-token semantic ambiguity. By accounting for that ambiguity before prompting for a determinate response we can increase the reliability of the output.
Two‑Step Contextual Enrichment (TSCE) is an experiment probing whether a high‑temperature “forced hallucination”, used as part of the system prompt in a second low temp pass, can reduce end-result hallucinations and tighten output variance in LLMs.
What I noticed:
In >4000 automated tests across GPT‑4o, GPT‑3.5‑turbo and Llama‑3, TSCE lifted task‑pass rates by 24 – 44 pp with < 0.5 s extra latency.
All logs & raw JSON are public for anyone who wants to replicate (or debunk) the findings.
Would love to hear from anyone doing something similar, I know other multi-pass prompting techniques exist but I think this is somewhat different.
Primarily because in the first step we purposefully instruct the LLM to not directly reference or respond to the user, building upon ideas like adversarial prompting.
I posted an early version of this paper but since then have run about 3100 additional tests using other models outside of GPT-3.5-turbo and Llama-3-8B, and updated the paper to reflect that.
Code MIT, paper CC-BY-4.0.
No comments yet.
