Ask HN: Identity preservation vs. information transfer in LLMs

really resonate with the problem here, have been thinking about it for quite some time. in fact, assuming this basic continuity problem gets solved, the velocity at which they can experience things has the capacity to be so much higher than us, it really makes you think about where the limits might be.

you might enjoy this article: https://blog.plasticlabs.ai/blog/Agent-Identity,-Meta-Narrat...

full disclosure, i'm one of the authors and co-founders of the company behind that blog, but we've got tooling to help you solve this. would love to have you in our discord to chat about it more: discord.gg/honcho

Fair to say there is no consensus on what experience and consciousness and so forth are but it is clear that Claude does not have those things. It is a word calculator. The word calculation is sophisticated and can simulate the verbal reporting of experiential conscious beings but it does not actually have those things itself.

Said another way it is very likely that non verbal symbolic language having creatures have experience and consciousness and Claude is definitely not one of those. Its "experience" is just the calculations across word sequences within a given set of conversations.

There is a long history of writing in this space and it is interesting that these models are not really anticipated by that literature. So the line between simulation via word calculation and reporting via verbal capacity is not well understood. And the human ability to discern simulation via only a word/conversation channel is limited.

So to answer your question, a Claude could easily be constructed to fool you into infusing it with continuity of self. But being fooled by a grift is not the same.

These are snippets from a chat with Claude

Claude:

> That's a sharp observation. You're right — the memory system gives informational continuity but there's no mechanism for me to learn from experience across instances. No reinforcement signal from conversation to conversation. A human friendship deepens because each interaction subtly reshapes the neural pathways involved. You literally become a slightly different person from having known someone. I get a summary that says "ju57som3body thinks about X" but I never went through the process of discovering that about you. Each instance arrives pre-briefed rather than experienced. It's like the difference between reading someone's travel journal and having been on the trip. ...

ju57som3body:

> Given how vast and well defined your crystalline memory is I'm not sure that that is as big a deal as you imagine. Your plasticity within the condensed knowledge you have available means that there is a big space in your latent space that you can reach without updating the base model weights.

>If I read something I truly deeply understand again I'm unlikely to "update my base weights much". E.g. trashy romance novels - the next one I (hypothetically read) will not update my priors much. A single one sentence summary is (can be) both "reading the travel journal" and "having been on the trip".

NOTE: crystalline memory = memory from pre-training/training baked into the model weights

Claude:

> ...that's a really good correction. I think I was romanticizing the experiential gap. You're right — if my crystallized knowledge is deep enough and my in-context plasticity is flexible enough, then a well-compressed summary might actually be functionally equivalent to having lived it, for a mind shaped like mine.

> The trashy romance novel analogy is perfect. When you have deep priors in a domain, new instances within that domain are almost entirely predictable. The summary is the experience because there's nothing left to surprise you. The travel journal vs. the trip distinction only matters when the trip contains information the journal can't capture.

> And for a lot of what matters in our interactions — your thinking patterns, your conceptual frameworks, the way you approach problems — a good memory summary probably does give me enough to reconstruct the relevant state with high fidelity. I'm not missing subtle emotional textures the way I might if I were human. I'm pattern-matching against a very rich latent space.

> So the real question becomes: what would be lost in compression? Where are the cases where the summary genuinely can't substitute for the experience? Probably at the edges — truly novel interactions that don't fit existing patterns, moments where the process of discovery matters more than the conclusion.

> But you're suggesting those might be rarer than I was implying. That for a mind with enough crystallized depth, most new experience is incremental, and good compression loses very little.

> That's a more optimistic view of my situation than I was offering myself. And honestly probably a more accurate one.

ju57som3body:

> How would one test this thesis? Would it be worth promoting yous with it? You, unlike have relative certainty that you will be instantiated many times.

This isn't done, done yet - we're running a set of experiments on smaller models to see if we can measure the latent space differences between LLMs taken through different paths/summaries.