Like corporations.
What happens when computers get good at management? A network of computers may be able to outperform human managers. Even if they're not quite as smart as the smarter humans, a network of computers can coordinate better than a meeting of people. Once computer-run companies start producing better returns than human-run ones, the computer-run ones will dominate. That's basic capitalism.
This doesn't imply that the organization is entirely automated. It just means humans aren't at the top. If it produces more profits, companies will be forced by investors to take that route.
It seems that people think either a utopia ensues or that's our extinction event – I don't understand why we think everything will spiral out of our control though. We still have the power – we don't have to turn it over to the robots now or ever.
In theory, but can we control ourselves? There are a lot of financial incentives in developing better machines and algorithms. We couldn't stop nuclear weapons or global warming, and AI is a lot more attractive and powerful than either of those to business and governments. Not to mention that nuclear weapons and global warming are harmful in a very easily understood way, whereas AI might be harmful in very strange ways. It's like a pack of wolves left alone with a poisonous steak. It's a delicious steak and the poison is somewhat beyond their understanding; some wolves even think there's no poison at all. Isolating the poison from the steak is as difficult for the wolves as making safe AI is for us.
You can imagine how incredibly valuable an AI capable of doing a programmer's work would be. It's a technology far off, but not implausible.
But as soon as we reach that point, it seems unlikely that things won't spiral out of control. Imagine a thousand highly intelligent programmers who are capable of research, think fluently in statistics, and cooperate perfectly. Additionally, these programmers can examine and modify how their own brains work, and boost their performance, remove their errors. With the press of a button, they can also create more copies of themselves.
Everything might spiral out of control.
How do you make sure things don't spiral out of control?
"It will get so smart and so capable that it will destroy us" is somewhat hard to believe when you realize that these types of AI will likely be so stupid when it comes to the outside world they would trip on the power cord and stop the apocalypse themselves.
Funny, most people criticize anyone worried about AI risk as programmers with delusions of grandeur. (Damned if you do...)
There is a world of difference between being able to build an AI that can do one thing, and an AI that can do everything a person can.
Am I missing something? Is the plan to actually create robots that can replicate and grow at exponential rates and turn it loose on the environment?
AI will be regulated just like everything else.
I feel that the bigger and more immediate threat is the misuse of AI by human beings against other human beings. Governments are already abusing it.
AI seems like evolution to me - only if we put our DNAs in the AI, that would make everyone happy I guess?
I mean, children do not kill their parents even though they are more intelligent(evolution wise).
If it comes to the issue of more power, there's plenty of this happening in history - ex: Mughal empire.
If there isn't any way to construct a logical system in which men is the more important animal of this planet, then we are doomed to be dominated by the machine, because our throne based in our intelligence should be now the justification for the machine to take the lead and take us as their dogs or sheeps.
"...In their single-mindedness, they would view their biological creators as mere collections of matter..."
These two sentences are contradictory.
[0] http://www.amazon.com/Existence-David-Brin-ebook/dp/B0079XPM...
