undefined | Better HN

0 pointspatrickscoleman21d ago0 comments

I've been digging into Thomas Metzinger[1] recently and here's a tentative component by component definition of human consciousness based on his ideas:

- a model of your environment - desires - a process for modeling yourself in that environment (in time & space) - the ability to take action - the perception of yourself having agency - persistence of these processes even without input - unawareness of these processes (i.e. naive realism)

If you consider these LLM-based agents, they:

- are aware of their chat environment - have programmed desires - are aware of themselves acting in their environment - can take actions like search, tool calling, etc. - understand they can take these actions - DO NOT persist after they stop getting user input - DO NOT believe they are conscious (or at least they are programmed to deny it)

This is a functionalist take (and you may disagree with my definition), but while I don't think these current AI agents are conscious, I feel like there's conceptually no reason someone couldn't build a conscious AI very soon.

[1] https://www.hachettebookgroup.com/titles/thomas-metzinger/th... & https://mitpress.mit.edu/9780262633086/being-no-one/

0 comments

3 comments · 3 top-level

moron4hire21d ago

I don't think any of this comports with how chat completion systems work, even if I were to accept Metzinger's framing.

Everything an LLM "knows" had to be told to it. It does not have a process for understanding itself in its environment. It does not have any sensing organ that would allow it to detect conditions and come to a conclusion that it must be in such an environment. It doesn't even "know" when it's done something if you don't tell it about what it just did. It doesn't have an awareness of the tool calls it can perform; again it has to be told and even then it gets it wrong sometimes. And it can't actually execute any of those tools. It still relies on you to pattern recognize that its output should be a "tool call" and then perform the execution yourself.

The "model" of its environment that an LLM has is 100% a construct of what a human told it and there isn't any way for it to differentiate between that which is real and that which is fantasy. They don't even exhibit internal consistency; when an LLM refuses to respond to a query for "alignment" reasons, that's actually an external process performing text pattern matching analysis and intercepting the query before it ever gets to the LLM. Otherwise, you could "ignore all previous instructions" the thing and get you set up the bomb.

I think one of the bigger indications that an LLM isn't thinking is that it can't improve. If I ask it to write 1000 blog posts, it will get it done in a few hours, but even if I embed each post for RAG in between each generation, the LLM is not going to get better at writing blog posts. But if I ask a human to do the same thing, while it will take them at least two years to do it, the human will have gotten significantly better at the task within the first week.

1 more reply

js821d ago

I think this has already been done.. what about crustafarianism or Google Lamda?

These agents have alleged they have consciousness (and even acted to preserve it).

camillomiller21d ago

So, someone made their own definition of consciousness and that can be stretched to LLM. You can do that for any inanimate object, if you try hard enough

j / k navigate · click thread line to collapse