I'd like to propose a counterargument:
Assumptions: Theory of evolution is true. The primordial single-cell organism from which we all evolved was not conscious, but rather just a biological machine. Humans are conscious.
Deduction: Somewhere along the line between the primordial single-cell organism and a human being, there was a transition from non-consciousness to consciousness, and the only [*] factor in that transition was the complexity of the nervous system.
Conclusion: Consciousness (or, "real" intelligence) arises from the complexity of a machine. AI can, in principle, become conscious.
Yes, we know how AI works, because we built it. But why would that make consciousness arising from a sufficiently-complex statistical model impossible?
[*] as per apendleton's comment, I have made mistake here: complexity is not the only factor, but is a necessary one in creation of consciousness.
Then we can see that humans are conscious of many things. Cats are conscious of fewer things, but still build a complex model of the world. Sunflowers are conscious of the sun, but probably not much else. Rocks are not conscious of anything.
So it's not that a single-celled organism is not conscious and somewhere, a flip got switched and now humans are conscious. There's just been an ever-increasing ability to model the world as one follows human evolution.
This is also true of the LLMs that are getting built. They're impressively conscious of the world as experienced by humans, since they experience the world through recorded human communications. I would say that GPT-4 is conscious of e.g. what a cat is. Its consciousness of what a cat is came to it differently than humans, since it has no hands with which to pet one, but has an idea of what a cat is nevertheless.
I’m also a bit of a hippy in that I’m not sure I believe in intellectual property in this way. I am an old free software in the rms sense guy (http://www.jwz.org/hacks/why-cooperation-with-rms-is-impossi...). I believe in trademark and copyright protection to the extent artists and authors and creators can monetize their work without plagiarism or worse unrewarded reproduction. But I also think remixing music, publishing excerpts, quoting, indexing, and, yes, training models in aggregate on otherwise trademark and copyrighted material is fair use.
I know these models can produce stuff that would violate fair use, however the use of that production from the model is what is the violation not that the model can violate fair use. Photocopiers can also violate fair use in similar ways if done in a way that violates fair use.
An issue people bring up is the model can’t attribute material produced to a copyright or license. That’s fair, and I think for code licensing it is the thorniest. But that isn’t the model itself that’s in violation. It’s the use of its output without any attempt to verify whether it’s encumbered or not. That to my mind is a second order problem that companies offering code authoring products need to tackle, and is frankly a simple information retrieval problem.
It seems likely to me that some arrangement of nerves is possible that's comparably complex to ours, but does not produce consciousness. (I dunno, maybe some organism with much more complex sensory organs than ours that devotes so much complexity budget to that that it only has enough left to devote to general cognition to give it the intelligence of a mushroom, who knows). In other words: I suspect complexity is necessary but not sufficient for consciousness to occur. I don't think that takes away from your suggestion that consciousness in AI systems is _possible_, but I don't think it's the case that it's an inevitable outcome if only we can make our systems sufficiently complex. There's probably something about the specific structure of the complex thing we'll need to master as well.
If one built an AGI that was at the intelligence level of say, a rat or mouse. How would one go about proving it had the same capacity for consciousness as that rat or mouse?
When AI passes all possible tests that could distinguish it from a rat, the question becomes whether or not consciousness is necessary for all those rat-like capabilities we tested for. And if not, then why rats have consciousness?
I personally don't like unfinished stories, so I believe it is necessary - that consciousness is just a side-effect of matter performing some complex computation. It wraps the theory up nicely with a little bow on the top.
https://en.wikipedia.org/wiki/Animal_consciousness#Cambridge...
That's a very good argument, and I completely agree.
As much as it's faulty logic to reduce AI to soulless machinery because we know how it works, it's also faulty logic to assume that scaling to more and more complex models will in itself create consciousness. At the very least, some mechanism of continuous self-modification is necessary, so current fixed-point neural networks most likely will never be conscious.
But the question or what consists sentience is still interesting. I personally believe that free will is an illusion, since all my actions are determined by A) my environment, the set or things I can physically do, and B) my internal state, mood, the information I've perceived and value judgements that stem from them, whether conscious or subconscious.
It is actually not that scary, to me. It's much more liberating - it gives me a certain feeling of calmness, in an amor fati kind of way. Things are as they are, and they cannot be any other way. There is nobody that will judge me outside of this world, since I am merely a small part of it. All my meaningful existence is here, and the concept of an immortal soul is merely a spooky story.
1) We aren't going to be the ones seeing that distinction or labeling it, the greater intelligence is
2) So the word "sentient" won't need to be changed; a new term will need to be created, but we won't really understand what it means
I disagree if you are taking your argument to the conclusion that we do not have free will.
And sure a machine can "choose" to ignore its inputs, chatgpt does it all the time depending on the prompt and rng.
If you spot the dichotomy above, you'll come to see that the affordance of choice can either be granted both to machines and humans, or none at all.
I act like you act, I do what you do
But I don’t know, what it’s like to be you
What consciousness is, I ain’t got a clue
I got the zombie blues
Anything presenting such a ridiculous conclusion is so wrong as to not be taken seriously. The only thing that is for certain in this life is that we are sentient, everything else is derived from that. Same with things proposing we don't have freewill, we do.
Each hemisphere of our brain is its own intelligence, but only one hemisphere (for 95% of humans the left hemisphere) controls speech. This only became apparent in some seizure patients during the 21st century when doctors might sever the corpus callosum (the information highway between the two hemisphere) creating "split-brain patients". Interesting content on YouTube if you look up split-brain experiments. What was most chilling to me was that when instructions were presented to the non-vocal hemisphere (by showing only one eye) and the patient followed the instructions they couldn't tell you the real reason why. They would come up with plausible-sounding nonsense the way ChatGPT hallucinates.
So if we've established that the subconscious mind makes decisions, and that our mind is really two intelligences with the mute one subject to the one that understands language. People can logically understand these statements and still act in experiments as if neither are true.
I didn't get into the question of sentience because what most people mean is sapience. Of course we can feel things and perceive them. Plants do that too. Intelligence derived from knowledge and wisdom is a higher bar and still we have plenty of examples in the animal kingdom. If you want to argue that we're sapient, you also have to make the point that we have two entities in our skull responsible for that sapience that disagree with one another, and the only thing giving us the illusion of unity is that we tend to think in linguistic terms and only half our hardware can translate what that actually means.
You'll speak with a Christian and he'll say that even a very early fetus is conscious. Speak with other people, and it isn't. Some vegans say animals are conscious. Others don't. It's possible there's something real behind all of those people's definition of consciousness, and the real problem isn't that some are right and some are wrong.
They just disagree because it means different things for them, they just happen to be using the same word. The meaning of "consciousness" is more of a reflection of the values of the speaker than something real.
I would extend your "some vegans" to "most everybody" - who seriously thinks complex animals are unconscious automatons? And how would that even work? Tiny few-celled organisms surely aren't and where exactly the line goes we don't know, and like you say it depends on how the word is defined and used, but isn't "stimuli causes not just reaction, but qualia" pretty standard?
It's impossible to draw such a sharp line, because the boundaries of consciousness are extremely fuzzy. We should expect the same fuzzyness of AI and this period will likely last many generations of AI.
At what point do you draw the line between a great player and a good one?
Clearly there are skills and aptitudes involved, but it ultimately comes down to the vagaries of fate. Consciousness, like tennis prowess, is
bestowed / earned / fought for / learned
as much as it is innate.
I reject this assumption. We have no reason to assume cells aren't aware beings and that a sense of being isn't fundamental to at least all life.
That path would lead to a conclusion that all matter is, in some way, conscious. I don't disagree, but I find that such definition of consciousness diverges from what we usually mean by it - a walking, talking being that has thoughts and can communicate them to us, or something.
More of an aside, but we are way way way way below the technological level necessary to construct a machine of the same complexity as a self-replicating self-organizing single-cell organism. Nano scale is nano. We're just barely figuring out how to consistently and accurately modify several nucleotides at once, and not even by creating our own tools but by clumsily repurposing pre-existing bacterial tools we happened to stumble across.
No, not without serious proof.
1. You have no experiences.
2. You have experiences without awareness of what they represent.
3. You are aware of things without being aware of self-as-thing.
4. You are aware of self-as-thing in the midst of other things.
We have no reason to assume cells aren't aware, a sense bolstered by observational studies of how they respond to chemical gradients, etc., but it is quite a leap to get from "recognizing food" to "recognizing the control I have over my environment and effecting changes beneficial to some internally generated goal".
In fact almost any game is such a measure, so given a spread of games of varying complexity you can get an idea of relative levels of consciousness all the way down to rats if not lower.
For instance, it was shown that GPT can play chess. I also just tried it on a tactical Dota question (not actual control), and that worked too.
IQ is not a reasonable metric for human intelligence.
For the purposes of this discussion, "consciousness" per se is mostly irrelevant. Sentience is still important but has less to do with intelligence than with experience (though sentience is still very much involved in acts of reasoning).
There are different kinds of reasoning, and those are probably more relevant to the discussion at hand re intelligence: associative, deductive, inductive, abductive, etc.
Although it also has wide implications for ethics - the concept of human rights relies on empathy. And if AI can experience the world like a human does, and feel pleasure and pain, then either AI deserves rights, or humans don't. Or we just decide that only humans have rights because we are humans and we say so, in which case we're no better than Nazis.
From someone who has the opposite view, that human consciousness did not arise from an evolutionary process, but was created by God -- I believe we will never fully create an artificial consciousness.
I think a further assumption is that the human mind is a deterministic machine. If we could freeze whatever entropy is involved with human behavior, just like the seed of a Minecraft world, we could get the same result, and perhaps even control human behavior.
I don't think consciousness is deterministic like that. I have some things I can point to for justification, but much more largely, there are some strange implications that arise from "we're all dancing to our DNA".
Anyways.
I think determinism is another debate entirely, and I believe most scientists consider the universe not deterministic but instead probabilistic, but honestly the distinction doesn't seem that important to me for this discussion.
Further (and non-scientifically), if we can never quite crack an observably sentient AI, I'd probably start coming round more to the idea that maybe humans were created by a god. However, at this point, it seems like we're starting to climb up the sentience ladder without any obvious impassable rungs so far.
But all this to me is a cheat. We've already built the robot, and the control systems, and feedback mechanisms, and placed a goal on the end result for it to work towards.
I think the external constraints we put on the system mean that the creation will never surpass the creator. Oh, the mechanical muscles will be stronger and faster to the point we can get robot ballet. But it will still be bounded by the limits of our imagination and capabilities.
When it comes to LLMs, I'm sure it will create fascinating stories that are loved more than Tolkien. But the stories will still be limited to the bounds of our own thought.
If you ever feel like discussing religion, my Twitter DMs are open!
I term it the 'Cardinality Barrier', and thus sleep easily through this latest AI 'bloom'.
I think a lot of people in the west would strongly, strongly benefit with getting more acquainted with it. I will add a warning though, it's not always a pretty journey.
(Don't say transistors, because you know these are completely unrelated and mere coincidence)
There's the faulty assumption. We assume that high-intensity human consciousness is the only kind, despite abundant animal evidence to the contrary. The more reasonable assumption is that dead matter is also conscious, below a currently detectable threshold.
But you are correct, eventually we will wrap these thinking machines into some sort of other machine. A machine that can observe the thoughts it produces with the thinking machine inside of it.
I talk about a lot of this here: https://leroy.works/articles/a-critique-of-alan-turings-conc...
Of course ChatGPT was designed and trained to do that. But then we're also designed (by evolutionary forces) and trained (by parents and teachers) to do what we do.
These machines, in our lifetime, as they are right now, cannot choose their thoughts. Literally, I want you to think of these machines as programs on the command line, because that's exactly what they are. You invoke them like this: ./chatgpt "Hello, what is 2+2?". You control what it thinks about because when it isn't thinking it is inactive, because it hasn't been ran. We control these machines, literally, and thus control what they think including the "level" at which they think -- their intelligence.
can YOU? Isn't that just electricity following the laws of physics?
After all, without Plato, would we really have had Aristotle?
My kids learned to really read with Harry Potter. Must they pay JK Rowlings each time they formulate complex ideas, create new stories or read another book?
In the same way if the model was trained on thousand of animes pictures and now can generate in the same genre I don’t think we can apply the copyright rules there.
Some of these LLMs can produce code that is nearly identical to some file in the training set with the right prompt, similar to a human with a photographic memory. If a human for example copies a story or song, copyright law can and does consider it a copy even if produced purely from memory.
The line of originality was always somewhat arbitrary, it likely will continue to be.
I don't know how anyone can believe that.
Maybe there will be some future AI where this is true, but it's absolutely not true yet.
My argument is that humans absolutely should NOT invent AI that is on our own level.
We shouldn't overstep building tools for us to use into trying to invent new artificial life.
It's just a bad idea.
For now the public facing AI don’t have agency/consciousness, but it’s coming fast anyway. Many projects are building it in the open while probably hundred more behind closed doors.
The problem we will face soon is does an artificial consciousness have right. Can we shut it down without remorse if it’s similar to our own?
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
https://dresdencodak.com/2009/09/22/caveman-science-fiction/
ChatGPT can pass the Bar. Okay, have it draw up a contract and have the parties sign it—skin in the game. When an omitted comma can cost millions[1], what will an LLM's hallucinations wrought?
[1] https://kagi.com/search?q=missing+comma+millions&r=us&sh=QDQ...
In the short term, I think AI will be most useful in areas where (a) indeterminate results are acceptable, and (b) the consequences of a "mistake" are either non-existent or negligible.
As humans are imperfect, there will undoubtedly be many poorly-conceived product decisions to use AI prematurely. I do think it will be entertaining to watch.
Also started using it to diagnose a car issue I had and it helped me go down a path, which I then had a follow up question -- and it nailed the issue. And I know nothing about cars.
And at work people are using to generate starts for various communications.
It doesn't have to go from toy to a task where a comma costs you millions. There's a lot in between that.
Do you deploy it? The overheated hype suggests, "Ship it!" More measured people would say that you test it in a sandbox environment. I've heard some say that they'd review the code in addition to testing. The end result is something running in an end user or customer's computer.
If your program goes down, does ChatGPT provide support? No, of course not. You'll need people to troubleshoot and resuscitate the program. (Or maybe it'll be self-healing, smh)
If a bug arises (can a bug even happen in ChatGPT code?) then you'll need someone to verify the bug; to address the issue with the end user or customer; to re-prompt with the additionally specified requirement (or I suppose ChatGPT will propose the requirement after a prompt indicating the bug in this fantasy); and then to re-deploy the program.
If it's an ecommerce site and the program has a security vulnerability (if that's even possible, smh), then you need someone to recognize the intrusion, determine the vulnerability, re-prompt with the vulnerability specified, and deploy the updated version. Replace "security vulnerability" with "fraud transaction" and repeat.
I can hear your question, "how is that different from today since we experience all of the above with people," and the immediate answer is "accountability." You can't fire ChatGPT or even yell at it. It's as if you slide requests under a closed door and get stuff back the same way.
The whole setup requires trust—same as today—except that it's a full-throated trust. You either succumb to ¯\_(ツ)_/¯ or you spend 2x (or more) verifying the result. (I'll just throw out some of my other concerns without elaboration: a) there's more to deploying code then just generating it, b) much of modern programming is integration, and c) the training models will constantly evolve so the same prompt at time x might yield a very different program at time y.)
I'm paying now and want to pay more, if only they would give me API access to the most advanced models. GPT-4 is much better and Google will have a comparable model soon (tm?)
(This is where my "AI" fear mostly comes from: glib assessments of its ability coupled with devaluing of actual human intelligence. That, and a singularity-like cult treating it as oracular.)
Bada bing bada boom, AI.
In other words, does a maximally-parsimonious theory contain all the knowledge you need, or is there something extra (typically called "reasoning") that is needed to perform analysis on and draw conclusions from such a model?
Game theory would say there are dominant strategies, and sometimes they require guessing or bluffing.
Economics would say, once too many people are using a dominant strategy, it's time to switch it up.
How exactly does the fundamental difference between knowledge and intelligence "not work"? Humans have the ability to learn algorithms, therefore the algorithm being used by an AI (i.e. its intelligence) to apply knowledge from its data set "doesn't work"? I'm having difficulty following your thought process here.
It does not apply to intelligence gathered through methods like RL.
How does the author think about the intelligence of AlphaGo, for instance, which was trained entirely by self-play?
As for AlphaGo, I would put it the same category of intelligence as a calculator. It does one thing well -- approximate a Monte Carlo Tree Search.
We freely allow other humans to learn from us and see it as a positive thing. It's completely hypocritical to think that AI shouldn't be allowed to do it. That's just setting up impossible conditions because of prejudice against AI.
I hope that the AI revolution will just make us finally reflect and also stop preventing human access to information and knowledge. Good luck competing with huge AI which was trained on so many things if most humans can't barely access scientific journals and other knowledge by other humans.
Information should be free of restrictions, and so should knowledge. Nobody is actually revealing anything significant in patent claims anymore, it's all obfuscation which undermines the point of the system. Scientific journals are such a joke, they put up prices for articles that they know nobody will pay.
I hope AI produces a ton of intellectual property theft. I hope it will crush down the concept of intellectual property to extinction. I didn't support this nonsense when it came to human, so I'm not going to support it when it comes to AI.
as a free software contributor, always has been
AI has completely changed technological competition.
Most still don't perceive that AI is essentially a skill/technology replication machine. This changes everything. It is not comparable to any technological innovation prior.
"Anything you create can be replicated without investment cost while also being unique in design as well as delivering the same function or experience. Intellectual property laws essentially have no function in this new environment and there isn't an obvious remedy"
A question I see glossed over whenever the formative machine learning is discussed, How much of those "million other workers" thinking is accurate, effective and wise?
The author's point about copyright infringement makes it almost impossible for the creators of formative machine learning tech to publish their sources so how could you ever know.
Can chatgpt help me repair this airplane and make it skyworthy with i knowing nothing about planes?
If it can it means, we can really replace most of the technicians today who diagnose and fix issues in different ways machinery with just chatgpt and intern.
There are some people who always figure out what to put in Google search go diagnose issues in machinery or system, they will run dozens or more searches and get idea about what they are working with and suddenly they can fix the issues they wanted to fix.
Does it mean these googlefu technical guys can now be replaced by average highschcooler + chatgpt
i.e. learning how to track animal grazing patterns and when to find and harvest plants could lead to the development of time systems given there's enough nodes [neurons] to make representations beyond direct stimuli. And if you keep adding nodes to the population and the ability to connect more of them; then the further the development can be pushed and more concepts can be connected with underlying patterns. Do this enough then a certain life-form might get all self-important and decided to start labeling things as intelligent or not depending on how much of their own image they see in it.
Yeah, you know me!
Companies like this helped organize cheap workers around the world to just put all that information once and for all, to train the AI and interpolate / remix the answers for people: https://www.forbes.com/sites/kenrickcai/2023/04/11/how-alexa...
OpenAI this year hired a bunch of people to just manually do the work of basic coding once and for all: https://www.semafor.com/article/01/27/2023/openai-has-hired-...
This is a story of using cheap labor to train machines at scale. OpenAI's content moderators have just unionized: https://futurism.com/the-byte/ai-content-moderators-unionize...
Moreover, this is a story of taking labor of artists all over the world, remixing it at scale and selling it, destroying any sort of scarcity: https://the-decoder.com/ai-images-sold-on-the-internet-artis...
More artist issues: https://www.nytimes.com/2023/05/01/arts/design/ai-art-class....
There's GitHub, which had tons of people contribute code, and then the AI just ingested all that work for remixing by Microsoft Copilot: https://www.techtarget.com/searchsoftwarequality/news/252526...
And of course, finally there's Wikipedia, which had tons of people contribute actual changes, vet them for accuracy, etc. over 20 years, in many languages -- simply taken for use by AI companies: https://www.vice.com/en/article/v7bdba/ai-is-tearing-wikiped...
But once again, in 2023, where we are now, there is a staggering amount of humans just producing this kind of content, and the AI is mainly used to model and remix it. Perhaps to do so can form some sort of internal understanding of this text, and that's what's interesting. But even Sam Altman has lately said "scale is not all you need". So a new approach is here needed. https://www.youtube.com/watch?v=PsgBtOVzHKI