After watching Legal Eagle, I asked a legal-ish questions about the Bricks and Minifigs case. Claude was outdated about the case and gave me some outdated info, so I tried to update it with the info I just saw online.
I updated by telling it I saw something in a LegalEagle video. It proceeded to tell me the video doesn't exist and I was hallucinating it, in a quite combative manner.
I provided a link and it insisted it didn't exist, with a quite verbose answer, once again very combative and arguing that I was talking in bad faith.
I provided a transcription from Youtube and it backtracked a bit but said I should have provided a transcription at the beginning of the conversation, since I knew the video existed.
I didn't say much to it, just a few sentences like "video is here: <youtube link>" and "I got its transcription: <pasted text>".
If you want it to synthesize information that is not in its training data (from a few months ago), you can ask it to research the topic. But, arguing with an LLM is like putting lipstick on a pig. Only the machine is incapable of becoming annoyed. It has infinite patience to continue being wrong forever.
Your mental model of what Claude is and does is the problem here. Short of a revolutionary breakthrough in AI techniques, the LLMs will continue to do matrix math across a huge bunch of weights that cannot change based on anything you say.
I guess putting lipstick on a pig might entail some wrestling, but it's a different idiom.
So, yes, mixed up idioms. But, the machine doesn't like arguing either. It is incapable of liking or disliking things.
I think you mean fableuous ;)
Sorry, but your mental model is wrong.
LLMs do matrix math across "a huge bunch of weights that cannot change based on anything you say", but the matrix math and results are informed (key concept here) by what you said, including the memory of what you said earlier in the discussion (and in some setups, even across discussions).
That's what a bloody prompt does.
It's entirely logic for the parent to want the LLM's matrix math + model + internal prompt, to accepts its prompt about LegalEagle and work with that, instead of arguing and giving him shit about it.
Especially since the earlier version of the model consistently worked like he wanted, and the new one consistently doesn't. He's not asking for some new unforeseen capability unknown to LLMs.
I provided a question, and when given an incomplete answer, I provided with more info.
It refused to accept the additional info due to limited access to Youtube.
There was nothing more than that. There were no expectations.
The hostility and the amount of assumptions here are very strange.
...almost as strange as having a website accuse me of hallucinating a video and trying to gaslight it :D
I asked Gemini Flash 3.5 through the Gemini app something that followed a similar pattern. I asked about something, it replied with outdated info, I said that's outdated, it did a web search and apologized for being wrong, then proceeded to give me good info.
That wasn't just a bare model, that was a model wrapped in a harness, driving the model and allowing for web searches for example.
GPT in Codex is even more aggressive, I often see it proactively do web searches to ensure it's not feeding me wrong info.
You also seem to be making a lot of assumptions about my understanding of the models, especially considering I just told a story :)
I never said anywhere I want it to learn or remember, or that I argued with it.
I just provided additional information to it (in the form of a dozen or so words, tops, per message) and it accused me of hallucinating and trying to gaslight it.
My messages never went beyond a dozen words or so.
very witty and very cynical, thank you
Well, they do think, in that they produce output that is indistinguisable from thinking. If a person produced the same output to the same questions, we'd considered them thinking, maybe dumb sometimes, or paranoid at others, but still a thinking person.
We can argue about the quality and depth of the thinking that LLMs do (and we can say it's much cruder than a human thinking architecture, and of course not real time), but an LLM quacks like a thinking duck and looks like a thinking duck.
It does not receive dopamine as a result for a good answer, and a split second after finishing your answer the very same GPU is probably translated french or something for someone in another state. This is a language generator which has a corpus of information and has been tuned to appear correct.
Even a dog will learn from recent stimuli, these things don’t. The prompt just modifies.
Basically the complaint is about how Claude is being trained.
You're so totally 1000% right about that, but they're really good at faking it, to such a degree that entirely too many people (even including some so-called "experts" in the field) have been utterly fooled by the mathematical "trickery" that performs the illusion of "intelligence".
I use another service for coding.
It's interesting how my experience there is mirrored by the answers here, though!!!
However, that doesn't apply when they are told to roleplay a scenario, so its easier to get it to accept and create output with the idea that this true fact you've seen is part of a fictional scenario, than for it to output the same words within the context of the fact being real.
As an aside, I don't that I have to personify AI in explanations and that all discussions revolve around anecdotes, but I only know enough about the maths behind it to be dangerous, not useful. Does anyone else feel this way?
