Also, many people seem to be mistaking "artificial intelligence" for "actual intelligence".
You know that these LLMs are not actually intelligent, right? Right???
They've had considerable success getting traction for these views with politicians in Europe, and that's the reason you see so much of this in the media currently. I'm hoping more sane voices will soon get organized, because significant parts of this are reminiscent of a well-funded doomsday cult.
I think this article is an early attempt at creating a politically viable counterpoint.
Maybe L2 systems. But those are just a driver assistance, the driver must supervise and intervene.
Actual L4/L5 systems haven't killed anyone as far as I know, except for the Uber accident, and even then the driver was supposed to be supervising it, I'm not sure if it counts as L4/L5
Got to hang out with some "hotshot" Facebook developers after hours. I remember Lee Byron (of React, GraphQL, etc) stating very confidently that the roads would be teeming with self driving cars by 2020.
Everyone wants to pose as the front(wo)man of that innovation, thought-leaders. Whatever.
It's all marketing, for their own careers, their own personal agenda.
The whole text looks like written by ChatGPT in a prompt like "Write about AI safety and relates it to the Age of Reason".
This is a good reason why AI should replace us, so we no longer have human beings writing that stuff to self-promote themselves.
The funniest part is in the end of the article:
"Eric Ries has been my close collaborator throughout the development of this article"
So the guy who wrote "The lean startup" is helping him with that article. drops mic
Can a submarine swim?
Does it matter if an LLM — which is far from the only kind of AI, and trivial to include in another more complex AI — does or doesn't match your definition of "intelligent" when it can write fluently in any language (human and code), and also getting near top-of-class results in the Bar exam and the biology olympiad?
Corporations — sometimes used as another example of an unaligned optimising agent — can't drive cars, they have to hire humans to do that for them, but corporations can still be prosecuted for dangerous chemicals (3M); and AI, just like other computer programs, can have dangerous output that you should not rely on (e.g. Thule early warning radar incident where someone forgot to tell it that it's OK for the moon to not respond to IFF pings and it nearly started WW3).
I think once you give AI working limbs and the ability to manipulate matter like humans, that's concerning, because if the AI becomes a runaway AI with no upper limits on the types of behavior allowed, we get into Paperclip Maximizer[0] territory very fast.
If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent."
Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. Assuming that this is not the case, don't hold machines to standards you're not willing to hold humans to.
This has to be a candidate for the most unhelpful statement possible. If you are sincere about it, then why not apply it to every word in your reply and see where that leaves you?
So you're saying humans, collectively, are gods? After all, as a species we started with nothing - no "clean" training data, no paintings to replicate, not even language itself. And here we are - arguing about whether one creation of ours, trained on a bunch of other stuff we created, is as intelligent as us.
So is all human knowledge. They're all things we "made up" to describe the world around us. That's a really cheap way to reduce something. There's a mountain of psychological literature that intends to describe and measure intelligence. Saying it doesn't mean much of anything is a bit of a postmodern hot take.
> If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent."
Painting pictures and composing poems aren't good ways to measure general intelligence. That's creativity. They're correlated but not the same.
There are specific tests for measuring general intelligence, and there's absolutely no way current gen LLM would score highly on them because they don't even accept visual input. Yes I've seen that they can score highly on verbal-only tests. That's obvious anyway because they have seen, written down, and have access to the answers during the test.
Even if they did accept visual input they would only be able to solve problems for which they've previously been given the answers (trained on the dataset). Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec.
An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. Poetry is inspired in a way that’s hard to replicate with a prompt.
Artists also take inspiration from other artists, but the “distance” between their influences and their work is so much greater than the distance between an LLM’s that I don’t fault anyone who think it’ll take time for LLMs to catch up.
The word was made up to cover a range of cognitive abilities that humans and animals (to varying degrees) possess. And we're gradually figuring out how to design machines with similar abilities. The general idea of intelligence being you can figure out how to do things that aren't just instinctual. a generalized intelligence can do this across any number of domains without some fixed limit.
Humans certainly have flaws when it comes to this. I've heard some discussions about the success and failures of AI in this regard. Can someone in this domain elaborate on the current state-of-the-art performance in this regard?
quality is required.
Meanwhile, I'm possibly one of the most Googleable people around (not due to fame, but due to nearly global name uniqueness) and it should be pretty plain to see that I'm a good actor (and kind of a big nerd) and that this might have all been a misunderstanding.
I'm kind of over it, but it is still hugely annoying. And it's terrifying to consider that no matter how anonymous you are on Reddit (and you might have very good reasons to be using an anonymous login), they in theory know exactly who you are.
What I'm saying is that AI could magnify a single human error in the past into an arguably massively unethical action; it could retroactively tie your anonymous actions in the past to your non-anonymous identity in the present. Where there is no escaping your past mistakes and you will always be punished for them, forever, by massive machines without any human intervention, or by systems in power that are arguably unethical and dystopian.
The one thing the Reddit Admin team (actual employees) takes seriously is people making new accounts to post in subreddits where they are banned.
Here's a random bit of fun prescience:
From Dune, by Frank Herbert:
> JIHAD, BUTLERIAN: (see also Great Revolt) — the crusade against computers, thinking machines, and conscious robots begun in 201 B.G. and concluded in 108 B.G. Its chief commandment remains in the O.C. Bible as “Thou shalt not make a machine in the likeness of a human mind.”
The usual interpretation is something along the lines that AIs took over the thinking for the leaders, making the leaders lazy, impotent, and society fragile and chaotic. This fits with a lot of Herbert's themes.
But, from Children of Dune:
> Three — Planetary feudalism remained in constant danger from a large technical class, but the effects of the Butlerian Jihad continued as a damper on technological excesses. Ixians, Tleilaxu, and a few scattered outer planets were the only possible threat in this regard, and they were planet-vulnerable to the combined wrath of the rest of the Imperium. The Butlerian Jihad would not be undone. Mechanized warfare required a large technical class. The Atreides Imperium had channeled this force into other pursuits. No large technical class existed unwatched. And the Empire remained safely feudalist, naturally, since that was the best social form for spreading over widely dispersed wild frontiers — new planets.
Arguable paints a different picture - that the AI in the hands of the masses would be too threatening to the status quo.
Oh certainly. Not for any higher purpose, but remember that OpenAI lost its shit and threatened to leave the moment the EU wanted to look into actually regulating the risks of their product. The driver is money, and little else.
AI people like Sam Altman want a very specific kind of "AI regulation" - a mostly toothless one meant to address the hypothetical of a Skynet (which is honestly still pretty far out there) rather than the more realistic problems that come from their product.
Things like IP law (these LLMs rely on scraping lots of copyrighted data), profiling people (in an age where privacy is important to a lot of people), medical malpractice (I remember someone saying they fed a patient dossier into ChatGPT on this site... Dear Lord I hope that person got shafted by HIPAA) are all genuine problems/risks with LLMs that don't come from the models themselves but from the people using them. Thats the real danger and OpenAI wants absolutely no regulation in that area because it could hurt their bottom line.
Hence why they also almost all tend to follow the beliefs of Elezier Yudkowsky, whose entire life is basically to claim Rokos Basilisk is totally happening and the only option is to give him (and his friends) lots of money to ensure it so you won't be tortured by the AI for not bringing it about.
Crowing about evil AIs is a useful distraction because it plays well populistically (Skynet, Matrix, HAL 9000 are all good examples of "AI out of control" in the public consciousness) and it gives them easy backing from people who don't understand how these things work. The EU wasn't impressed and the AI act they passed addresses some actually meaningful concerns. It remains to see if US Congress/Senate will address it at all or if their business interests reign supreme.
That is 100% what the whole AI panic is about.
2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong.
I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day.
I am happy politicians and think-tanks are taking this seriously, you should be too.
The domains where AI experts can beat domain experts is certainly growing, but I don't see how you can get from that to a claim of general AI. I certainly don't see how you can get there from the recent LLMs in particular which can't beat domain intermediates at anything.
The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.
I would be remiss in saying they are anywhere near average human capability in most areas, but I do worry that my estimation capability is off and that we're closer to the concave part of the S curve than the convex. It just takes a couple more breakthroughs in model/dataset design
It's not just about language. Language models can predict novel protein structures, no folding required. https://www.nature.com/articles/s41587-022-01618-2
>What is the roadmap to get to general purpose AI, and what is the proof that we are close?
What testable definition of General Intelligence does GPT-4 fail that a big chunk of humans don't also fail ?
What's your working definition of "general purpose AI"? What does it include? What does it not include?
Does the AI have to be smart enough not to very often state incorrect information? LLMs still are unimpressive to me from that front.
I asked ChatGPT to explain the usage of the term "Wonder Kid" in the popular Apple TV+ series Ted Lasso, and which character it's tied with. It gave me confidently-stated incorrect answers multiple times until I told it the answer. (At which point it stated the correct answer with an apology.)
This doesn't feel close to "general purpose AI". Feels like we're meeting the limits of LLMs and finding they're not a magic AI bullet.
Textbooks are all you need is another extremely interesting area of focus.
Improved correctness and a general reduction in hallucinations I predict will end up hitting OSS models by EOY
(Shrug) If it turns out that humans have a higher purpose than doing a robot's job badly, I don't see the downside to letting the robots do the job. I'm all for capitalism, but it's an ideology, not a religion. If there's something better on the horizon, let's help it along.
ML looks like it is probably going to be the next step in human evolution by punctuated equilibrium. And it's about time.
What we've discovered is that the first model breakthroughs have been able to largely demolish that dream. As a writer I can tell you that writing with GPT4 today provides me with superpowers that non- accelerated authors could come close to; and theres no plausible way to distinguish our two works.
I'm more worried about the coming depression / suicide spiral (which were already in by the way) because of human uselessness than I am of terminators coming to farm us.
Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world.
I foresee a depression/suicide spiral never before seen in our history unless we do something radical
Assuming you have 200 IQ, if you could make thousands or millions of clones of yourself quite cheaply, you still might not succeed in taking over the world, but it is no longer a laughable idea.
Overall, I’m somewhat ambivalent about the possibility of x-risk, especially if we work reasonably hard to prevent it.
But it shouldn’t be ignored. AI is moving very quickly and it is unclear how powerful its capabilities will be in the next 10-20 years. Of course, there are many other risks presented by AI that we need to stay on top of as well.
Copying AIs just results in effectively higher intelligence (bigger brain). Things would be different with self-replicating robots of course, but duplicating robots is not that fast and one could argue that the robots are dangerous rather than the AI controlling them, because they would be dangerous even if controlled by stupid non-AI program.
How about 500 iq with a direct line into the entire information retrieval and processing system of 7 billion humans, most of whom are already easily manipulated by relatively rudimentary social media ML.
It's not about 200 IQ, it's about when we reach bigger gaps than that. Though yes, I suspect if the 200 IQ guy can copy himself at will, get hardware to do years of 200IQ thinking in an hour, etc. he might succeed.
The hairless monkeys were embodied and violent and they still took millions of years to dominate. The question is whether intelligence on its own provides sufficient advantage to be immediately dangerous. How can you conquer the world by doing "years of 200IQ thinking in an hour"?
I wish I were kidding.
This is entirely false, sounds like Andersen that seems incapable of understanding that agency and goals will be a thing.
We should focus more on the making infrastructure safe from abuse and not how to cripple AI models for them to not realize how to do harm. More like 'How to ensure humans don't drop atomic bombs on each other' rather than 'How to make atomic bombs safer to drop on people'.
Humans are too fucking lazy to actually accomplish this. We hook more and more internet connected systems to things with a digitally controlled motor every day with things like "#disable in production, set for testing, disabling 2FA to..." that then get pushed to the customer.
Fly some airplanes into some skyscrapers, etc.
Nor do they now, of course.
Can you prove this?
The existential threat argument is a ridiculous notion to begin with. Pretending open source LLMs don’t already exist makes the argument even more silly.
https://twitter.com/jeremyphoward/status/1678558165712113664
Whether it's a convincing argument is different question, but it's not for lack of knowledge about the state of art in AI.
AI is already increasingly helpful to AI researchers. So one convincing argument for existential risk is that if this increase continues we could see exponential growth in the efficacy of AI research. Given that the goal of AI research is useful intelligence, it seems reasonable to say that an exponential increase in AI research will lead to an at least linear increase in useful intelligence.
Another argument is that there are breakthrough discoveries that will dramatically increase useful intelligence, or dramatically increase the rate of increase. We've had some of these so far, so it seems likely there are others awaiting us. And it's impossible to say with any certainty that there's no chain of such discoveries that doesn't lead to a human or superhuman level of intelligence. All we can do is try to estimate the odds of it happening.
Both arguments seem a pretty clear argument for an existential threat, because as soon as you have even just a human level intelligence integrated that seamlessly with the power of conventional computing, you have something that is much more powerful than most (if not all) human organizations, and once that power exists, there's a non zero risk that it will aim at some objective that is harmful to us, in a comprehensive enough way that it also aims to stop us from being able to stop it.
I wouldn’t say dangerous AI is out of the realm of possibilities, only that we are far from a point technologically where it makes sense to have the discussion. It would be like speculating about the dangers of nuclear energy before discovering the atom.
Our current models do not pose an existential threat to humanity. Period. And yet here we are, discussing the merits of banning open source LLMs. Language modeling is genuinely useful. I’m worried that the AI hysteria will push us toward a future where the best models are needlessly regulated and controlled by corporations.
Incorrect. Power comes with the ability to command resources, not intelligence. The United States military is the most powerful human organization for its ability to marshal the most amount of deadly weapons. Some rinky dink AI company cannot do this.
But what happens if regulation makes the use of these models more legally dangerous than a few "well regulated" models approved by some sanctioning entity?
https://www.theverge.com/2023/7/9/23788741/sarah-silverman-o...
Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it.
That's one of the main business models of the Internet, only usually it's not even scraped. People upload it willingly in exchange for free hosting and free social media.
The other big business model of the Internet is mass surveillance driven targeted advertising.
And if someone hosts user-submitted copyrighted information without a license, the copyright holder can submit a complaint via DCMA and the hoster has safe harbor from liability.
OpenAI is literally scraping copyrighted works, packing it up and selling it without a license. Art, books, magazines, everything. No safe harbor from DMCA for doing that.
We wish. AI is hardware-limited and hardware is not moving fast. We are very, very far from matching raw compute power of the human brain. Robots are even more limited compared to human body.
AFAIK the biggest limitation of von Neumann computers is memory bandwidth. Brain is moving 10+ PB/s between neurons. This does not count synapse-local and intra-neuron bandwidth.
But that's not an apples-to-apples comparison, as the artificial neurons of neural networks are just a rough approximation of our neurons. They are also connected in a much simpler way too (nicely divided into layers, where each layer can only pass signals to the very next layer), so it could be that it isn't just a matter of how many parameters you have.
But the one that was just released about AI for autonomous drones, is pretty shocking.
The military side is farther along than I thought, AND, you know what is appearing in a Netflix documentary is old, it isn't the latest things that are still secret.
In any case, why it relates to this article.
We are in a race with other nation states, so all of this discussion about 'curtailing' or limiting AI, is all just grand standing, smoke screen.
Nothing will slow down development because other countries are also developing AI, and everyone firmly believes they have to be first or will be wiped out. Hence no one will slow down.
PS: I don't thik AI will pose an existential risk to our species anytime soon, if ever. But it can already cause damages if used improperly, like any other technology. So it has to be regulated, like any other technology.
Nuclear weapons are a risk to everyone, so the world(US, Russia) did come together in various treaties to limit production, development, etc... So, this could apply to AI also, everyone sees the risk and comes together to curb their us and development.
On other hand:
Nukes are very difficult to make. Iran and North Korea do not abide by the UN resolutions, they are continuing trying to develop them. We are just lucky that is not so easy.
With AI, the barrier to entry is much lower. You can find things that can be weaponized for free on github. It isn't like they are going to bother reading the licenses and not use them.
And for compute power, sure, they have harder time getting the best latest CPU's, and there is an embargo. But still, I think, far easier to get illegal gotten CPU's, than developing Nukes, the barrier is still lower.
Yes, hence the whole “doom”
The capabilities of current generation AI—LLMs + audio/image/video generative models—are far enough already that wide-scale distribution is extremely dangerous.
Social media allowed for broadcasting from 1:many and 1:1 with troll armies, but LLMs are on a whole other planet of misinformation. A scalable natural language interface can target people at the one-to-one conversation level and generate misinformation ad hoc across text, voice, images, and video.
The trouble with this approach is that harm will rack up way faster than liability can catch up, with irreversible and sometimes immeasurable long term effects. We have yet to contend with the downstream effects of social media and engagement hacking, for example, and suing Facebook out of existence wouldn't put a dent in it.
Current generation AI has enough capabilities to swing the 2024 US presidential election. It can be used today for ransom pleas trained on your childrens' instagram posts. It doesn't seem farfetched that AI could start a war. Not because it's SkyNet, but because we've put together the tools to influence already-powerful-enough human forces into civilization-level catastrophe.
The only reason an LLM would even have such access to people to do the hyperbolic damage you claim is thru ad networks and social media anyways - yet you are calling to behead public access to tech instead of regulating propaganda platforms. Both seem facile but your selection is the one that seems subservient
I wasn't convinced, but I did the maths and, well, looks like you're right:
Hiring people in to read and respond in a politically biased way to each person in the USA would take, let's say 2 messages per day for a month, so 60 messages per person, let's say 100 million actual voters you care about so 6 billion messages, let's say 15 seconds per response so 25 million hours, let's say you're hiring Kenyans because I happen to have the GDP/capita and workforce participation rate to hand, so it's about 2 USD/hour on average, so 50 million USD.
With GPT-4, and let's approximate each message as 100 input tokens and 100 output tokens, that's 0.1 * ($0.03 + $0.06) = $0.009, times the same 6e9 messages = 54 million USD.
So, yeah, looks like you're right.
3.5 is cheaper, but tends to be noticeable; from what I've been hearing (fast moving target and all that) the OSS LLMs tend to be 90% of the quality of 3.5.
You're conflating a communications channel with a new technology that enables abuse (as well as positive outcomes) at a previously-unimagined scale.
Is it not facile and subservient to default to the ostrich algorithm for new and world-changing technologies?
https://www.lesswrong.com/posts/6untaSPpsocmkS7Z3/ways-i-exp...
> Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways
> There are interventions we can make now, including the regulation of “high-risk applications” proposed in the EU AI Act. By regulating applications we focus on real harms and can make those most responsible directly liable. Another useful approach in the AI Act is to regulate disclosure, to ensure that those using models have the information they need to use them appropriately.
Power imbalance is not 'much worse' than extinction.
It's very clear how people start with the conclusion - "openess and progress have been good and will keep being the best choice" and then just contort their thinking and arguments to match the conclusion without really critically examining the counter arguments.
Okay, GLHF.
This is more utopian than communism even.
"There will still be Bad Guys looking to use them to hurt others or unjustly enrich themselves. But most people are not Bad Guys"
Failure to understand that even one by guy can kill everyone with a sufficiently advanced AGI
