Slightly more seriously, you could perhaps make an argument that, just like weight decay, an apparent "anti-contribution" moves the learning trajectory along, and helps the network settle into a more optimal basin eventually.
That way, my contribution is still valuable on the net, and I'm owed $0.00000003 positive dollars instead.
I thought the reason was the "reasoning" didn't work very well with "aligned" model output, so they had to remove the alignment during reasoning and then hide it to avoid exposing "unaligned" model output.
Before the massive nerf (showing summaries and suppressing certain aspects of reasoning) you would literally see reasoning text appearing on your screen like “while xyz is true, these facts may be seen as supporting hateful rhetoric or a conspiracy theory which is against my policy guidelines. i should tell the user xyz is not true or steer the conversation in a different direction. according to my instructions misleading the user is permitted in certain contexts where sensitive information is being discussed or could cause liability”
They disabled it shortly after the first screenshots appeared online, and restored it the next day in a way that hid what was actually happening.
There's nothing in the reasoning tokens that'll give bad publicity that the final output already wouldn't do.
I think one of the reasons could be to limit liability too.
What if reasoning helps in establishing provenance for questionable sources ?
What if reasoning and model's "thought" points to fundamental issues in how the model was trained to produce certain problematic responses ?
f we want more useful products, we need to come up with ways to disincentivize this behavior. Even if doing so poses an existential risk, we are better off if companies taking existential risks to please us is a necessary being a top player in this game.
https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...
It’s quite interesting to read. I can’t imagine using a model like this without the ability to peek inside and see if it is getting stuck.
You are correct in my intentions on this post generally.
I want to highlight:
I want to measure performance of the LLMs over time- which includes assessing the quality of their outputs. I don’t perceive the reasoning output to be anything other than a measurable signal of possible drift in model performance.
Except it isn’t, because I’m only getting a low value summary of the thinking.
It’s like asking your buddy how fast he thought that last pitch was when radar guns are behind the plate.
Yeah, it’s a description related to what happened, but it’s not the thing I want to measure.
It only makes sense that the same mechanism comes into play in strictly-verbal contexts.
Also, this is why "distillation attacks" are largely bullshit that Anthropic spreads for political purposes. Proper distillation requires access to the logits.
Why do you need logits? Can't you just train on cross-entropy loss of the model against the hard decision, like you do in regular pretraining?
There are definitely current-gen open-weight models (Step 3.7 Flash is one) that refer to themselves as an OpenAI model in CoT, but not in the final response.
If that is the case thinking is not visible to us as users due to it not being done in text.
Idea somewhat similar to what you describe exist but they make steering/post-training/interpretation much harder.
EDIT:
They link to a Meta paper from 2024/2025 though: https://arxiv.org/pdf/2412.06769/.
I don't know about Claude, but latest GPT versions still have a readable reasoning stream. It sometimes leaks out when the model gets confused, e.g., during a tool call. If you're curious, looks simplified; less words; extremely compact. They optimize tokens. But remain readable.
This could all be optics as well to try to give the appearance of a defensible moat. E.g. they can claim to investors that they are able to protect a significant chunk of their intellectual property this way. I'm not sure if anyone has a study about how significant the summarization is to distillation.
In the case of makers of open-source models (which are also competition), there is no allegedly, they were (and still are) openly doing that.
You've got that backwards, .bmp is a lossless format and .jpeg is the lossy one.
Fun fact: if you go back to the old school from 2 years ago and provide explicit CoT prompts, you get the full thinking prompts back again!
So you disable thinking altogether, and instead make thinking part of the regular prompt by prompting it:
“Before providing your answer, think step by step. For example:
The use is asking me to… I need to think about the blah blah. First, I should foo the bar, and then blah blah.
Answer: <put your final answer here>”
And tada.wav we have CoT as it worked in the GPT3 era back again.
Still, one of the daily most played WAV files worldwide, Id guess? :-D
Humans somewhat do the same - something that's been demonstrated in split-brain experiments.
Because of the nature of how LLMs work — text prediction engines - by putting the explicit reasoning steps first, it improves the likelihood of the final answer (which then is being predicted based on the entire reasoning chain as input) being correct.
1. https://medium.com/@eshvargb/the-llm-journey-how-neural-netw...
This evades an easy yes or no, so:
1. Many consumers believe reasoning-models allow that kind of question to be truthfully-answered, and their belief it reasonable given the marketing going on.
2. Implementers probably do not have the same belief when it comes to the terms mean or what capabilities they imply.
3. Yes, it doesn't actually do what the customer wanted it to do, which is a kind of retrospective introspection of internal thoughts and ideas.
____________
I advocate looking at everything from a document-generation perspective to cut down on traps and cognitive illusions. The "reasoning" models are a change in the style of document being iteratively-grown by the LLM, as opposed to something more anthropomorphized.
* Default: There's just the spoken dialogue between a Human Customer and Helpful Chatbot.
* "Reasoning": There's the spoken dialogue and a bunch of times the Helpful Chatbot character has an internal monologue. This provides more consistency between iterations, and can be mined by custom tools to call external code and insert results.
If your Human Customer character ask "Why did you say that", the LLM does not engage in a different process than "I have eaten an apple."
The LLM has no memories to consult or hidden goals to contemplate, it's the same process of finding more stuff that fits at the end of the document. Any benefits from a "reasoning model" is the LLM generates much better-looking additions because there's more (hidden) stuff for it to confabulate against.
- "Read `description` and create a specification, implementation guide, and checklist." - "Ask clarifying questions. If any of those questions has a clear best recommendation, please select that yourself and record that in "autorecommendations.md". - "Have codex and antigravity review each of these and work to consensus."
These are the core of ~61 lines of prompting I do across 3 prompts, and I feel like the resulting artifacts describe some of the thinking. Also, some of the back-and-forth between the models feels like it gives some insight into the model "thinking".
I will say: I heavily used Fable when it was available; Opus + loops + codex and/or antigravity review is better than Fable at building things.
Mind sharing your prompts?
If it's useful, it's useful, enjoy. If you aren't comfortable with that, don't use LLMs. You aren't going to get a mathematical proof of your output, just learn to be comfortable with that, or opt out and be a goat farmer.
No, they aren't a summary. They are the actual decoding of the sequence of tokens emitted during the the “thinking” stage of response generation.
Just as with, say, a human onner monolog in words vs actual speech, they are a product of the same output process as the non-thinking tokens. They aren’t a translation of the internal process that precedes the output mapped into language, either as a full result or a summary.
Having access to the reasoning text and output would help with performance measurement.
For daily use I actually like the reasoning summary to be brief/quick to scan.
That said, I understand the author’s desire for the real thing. It just feels better to have that access, especially when Anthropic will give it to you, but encrypted.
(Dimethyl(oxo)-lambda6-sulfa雰囲idine)methane donate a CH2rola group occurs in reaction, Practisingproduct transition vs adds this.to productmodule. Indeed"come tally said Frederick would have 10 +1 =11 carbons. So answer q Edina is11.
0: https://wiki.roshangeorge.dev/w/Blog/2025-10-12/Word_Magic#I...?
1: In the sense of true belief, I suppose
Yes, several models think in weird jargon. Here is an example of Mythos's thinking while playing solitaire: https://www.lesswrong.com/posts/wCSEpT3dTGz4N86Wi/even-illeg...
> 7♣-removal-IS-the-prerequisite-for-10♠/9♥!!)-⟹-OVERLAP-(ii)+(iv):-{6♠ J♦ 9♥ 2♣}-=-FOUR--—-UNLESS-7♣'s-seat-8♥-...-and-2♣-drains-only-at-crack-:-⟹-2♣-celled-+-9♥-celled-simultaneously-UNAVOIDABLE-in-t8-dig--—-BREAK:-9♥
This is a small step in the direction of something called "neuralese", where the model has stopped thinking in English and is thinking in internal vector spaces. Since this gets serialized through text, it isn't quite true neuralese, but it's moving in that direction.
I mean, I'm sympathetic towards the models. My internal thought process when writing code uses lots of intermediate steps that would be hard to write out in English.
fyi openai does the same; not really surprising or particularly evil
Nope, not your agent, if you're not running it locally. You just get to use it in whatever way they allow (also see the whole OpenClaw backlash and claude -p changes), unless there'd be regulation and laws around this (which there aren't and would be lobbied against anyways).
> Getting the full thinking output requires an enterprise agreement.
If you truly need it, then that's a (costly) option. Seems like they're largely doing this to prevent other AI foundries from doing as much distillation and stealing their CoT output en masse.
Luckily more open models don't generally do that.
Edit: If you still need something decently capable in the cloud, I’d suggest GLM, DeepSeek, MiMo or Kimi or Minimax, maaaybe sometimes Mistral for a simple EU subscription. Or look at all the pay-per-token options on OpenRouter, though be mindful of quantization.
For running something locally Qwen 3.6 35B A3B is presently a decent starting point but it will be rather limited, either way you can look up the Unsloth quants on HuggingFace for something like llama.cpp or Ollama or LM Studio.
All will work with OpenCode and Kilo Code, and most other tools. Can also try with Claude Code, I made a tool for that too: https://ccode.kronis.dev/ (or just set the env variables and maybe some aliases for something close enough), but frankly OpenCode is nice nowadays.
Proprietary technology is fun /s
What a waste of time
Well yes exactly, because they have billions of investments riding on it and why would anyone semi-bankrupt their org paying API rates for Anthropic, if a hypothetical DeepSeek V5 Pro would have almost all of Opus capabilities at that point, due to immense distillation?
Tell me this. If you hired a junior engineer or designer who refused to explain their thinking on their code and how they solved for the spec what would you do?
(That being said the reasoning output is still a summary of the Kvcache)
Any explanation that someone gives of their thinking process is necessarily lossy and likely partially confabulated.
You cant even guarantee WHAT model you get. Or if they downgrade you. Or if you 'offend corporate sensibilities' and they misdirect or lie.
The only way to get good returns on a model is to run it yourself. Quit paying for corporate bullshit.
Nor does knee jerk accusation of "anthropomorphizing" negate the fact that procedures that mimic human processing, even when done in software, are deservingly anthropomorphized, because they're a legitimate approximation of the human equivalent operations.
https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...
https://www.researchgate.net/publication/316045349_Post_Hoc_...
I'm not sure that applies to discursive writing, when we essentially use rules of logic to decide on the course of the narrative. Non-verbal heuristics still applies, of course, but we constrain it, so it's probably not entirely post hoc.
Pages of “I have to be careful, the user is asking that I do something related to cybersecurity that could easily be turned around and used offensively” but then happily gives me what I wanted.
> preventing misuse.
Imagine not being able to read the tokens you are paying for.
writes this^ and then proceeds to highlight a bold title from the docs that says "summarized thinking" that explains things clearly in the first sentence. lol
Interleaved reasoning and function calling makes this even more dangerous. A model can call functions during the hidden reasoning phase. An attacker could then exfiltrate data from you while the reasoning summary hides it from the user.
It also makes it impossible to know if the model is doomplooping during reasoning and burning tokens for no reason, as gemini is want to do, which we know about because its hidden reasoning often leaks out when it doomloops.
When the models are AGI and secure from prompt injection I may stop caring, until then I want to know exactly what the model responds to my prompts. or exactly what the agent is doing on my behalf.
Edit, further reading: Fooling around with encrypted reasoning blobs https://blog.cryptographyengineering.com/2026/05/29/fooling-...
Setting aside coding agents.. we really need this information to even pretend to evaluate the claims of stuff like mathematical breakthroughs, which is exactly why we will never see it. Very embarrassing to get the right answer for the wrong reason. But to give the models some credit, you could argue that even paying too much attention to the thinking is misunderstanding how CoT works. The argument would be that thinking in LLMs isn't really thinking, that it's self-reinforcement and circling to to encourage stability around beneficial attractors instead of degenerate ones. Can't have it both ways though: either the thinking is thinking and so it should be correct. Or the thinking is NOT thinking, and it's NOT real justification for the outcome, and these systems are even more hopelessly opaque than we usually assume.
Back when I used antigravity, it used to show the reasoning intact - at least for Gemini Pro 3.1, and likely for Claude Opus 4.6 (not 100% certain about it). I have some recollection of stopping the models mid-turn when they started going astray.
As a power user, I find reasoning fascinating to read and genuinely useful at times. Probably not that useful for 80% of their base.
The LLM providers will clearly evolve to be more and more opaque as their services get more capable. The frontier models may even be provided as purely internal advisor or async only so they can monitor your CoT and final answers for cyber etc.
RL (the basis of LLM "thinking") is a pretty crude way to achieve the appearance of reasoning given that it reinforces all the steps, including missteps, that got it to a reward. Providing a summary could be seen as form of sane-washing, making the model look more purposeful and directed than it really is!
I do miss the days when reasoning was visible. Another point for open source models!
> You've provided the current rewritten thinking and the guidelines, but I don't see the "next thinking" content that I should be rewriting. Could you provide the next thinking that needs to be rewritten?
These sentences are completely unrelated to the actual conservation
It’s much harder to understand _why_ a model chose a particular approach in Claude Code. Especially because Claude will happily give you hallucinated reasons if you ask in retrospect.
Recent anecdote:
I was reviewing a colleague’s PR and Opus 4.8 decided to write the new feature in a completely new module. It was unnecessarily complex. We had a hard time understanding why it chose that, and it told us that it was so we could eventually deploy it as a separate micro-service and test it independently. What?
Only after being more a lot more specific about the implementation and spending a lot more tokens, it flat out refused to simplify the code with the actual reason. It turns out a line recently added to CLAUDE.md was making it incorrectly think that the module it was originally supposed to modify was legacy code that it was forbidden to extend.
This would have been caught immediately if we could inspect its thinking process.
1. make distillation much harder
2. safety: prevent modifications to the thinking leading to injection attacks.
3. also honestly sometimes the model raw thoughts can be deranged and is not a good user experience (consider the varied audience in the market, etc.)
also often the mass underestimate/the model makers over-estimate how people love distilling models
this is really really not that bad at all
In further reflection it is such a great indignity & such a collosal barrier to working with the machine that it insists on being a black box. The disingenuity of the American models (small print: except AI2 & some other labs; you all are so great) is a massive disadvantage to their use,... and a massive slap in the face.
It's a threat to human intelligence that it is not co-participative. Walking further into my own judgement and feelings: the insistence on being an opaque black box, the Seals Chinese Room, is such a vicious harm to society! This is civilizationally an unsafe form of AI that probably should be outlawed as anti-social. It's an impermissible asymmetry, a crippling dependent relationship to be forced into. I'm working myself up, but here: this.. imo, this is not just indignity, is harmful, it is evil.
This "6 month behind" trend we've seen for open models feels like at some point will be less important than simply the models unwillingness to speak for itself & to be observable.
I suspect that in some decades, as other architectures are found and used, that the inability of an LLM to "think" without also emitting a token will be seen as one of their fundamental limitations.
> The computation we can see looks like it’s just guessing the answer, despite the chain of thought suggesting it’s computed it using a calculator.
It might be hallucinating or lying, it's not like you are actually observing the internals of the model.
Computers don’t think they process, those are very different activities.