I had this exact thought the other day in another context.
Lately I've been trying to puzzle out why there's been this outbreak of seemingly absurd and ridiculous nail biting over artificial intelligence in and around Silicon Valley circles. Rationally it makes little sense.
If you don't know, I am referring to this kind of thing: http://blog.samaltman.com/machine-intelligence-part-2
We have no evidence that "Hollywood AI" is nigh, no evidence it will "explode" and become super-human in a short period of time (and some very good counter-arguments against this scenario), and no evidence it would be intrinsically more dangerous than we are to each other. The whole fear mongering topic seems rooted in a tower of speculations that becomes increasingly precarious as you ascend.
I wrote a maybe 3/4 of the way baked blog post on it here: http://adamierymenko.com/on-the-imminence-and-danger-of-ai/
That blog post addresses some of the issues such as whether AI can or will "explode," but to me it felt like I was still struggling with the ultimate question of what really lies behind all this. Then maybe yesterday or the day before I realized that these fears might be rooted in the fear of disruption.
Consider Francis Fukuyama's very similar -- and perhaps equally shaky -- fear-mongering about transhumanism.
http://reason.com/archives/2004/08/25/transhumanism-the-most...
So transhumanism, which is basically the nebulous idea that we should attempt to radically improve ourselves, is what Fukuyama thinks is the most dangerous idea to future human welfare? Really? I can think of a few concerns, but how is this more dangerous than other much more obvious candidates like religious fundamentalism, totalitarian nationalism, or certain varieties of misanthropic nihilism? You know, ideas already drenched in blood that seem to have a disturbing ability to recur throughout history?
Fukuyama is also well known as the author of "The End of History," which is basically a court intellectual feel-good tome assuring today's leaders that the world has achieved a steady state and nothing much is going to change. (It's since become a laughingstock, as it should have been on the basis of its absurd title.)
Perhaps what scares certain people so much about AI is its potential to upset the world order. Human systems of control and authority are largely based on the systematic exploitation of human cognitive biases and fallacies. Even if an AI weren't explosively super-human, it might still operate in ways that are non-human. In so doing it might simply not be vulnerable to the same techniques of persuasion. How exactly does one rule aliens?
Maybe the fear isn't so much that AI is going to kill us all (especially since it would probably be symbiotic with us), but that it'd be a loose cannon on the deck.
At the same time, even a non-sentient but very versatile and powerful AI -- a programmable "philosophical zombie" if you will -- could obsolete entire industries overnight. As the article says, capitalist economies can cope with some amount of so-called creative destruction but too much is bad news. What happens if/when some kind of AI can do >50% of the job of lawyers, doctors, politicians, journalists, non-fiction writers, bankers/financiers, etc.? You'd have wave upon wave of bankruptcies both personal and corporate.
A real deep and wide breakthrough in AI could be hyperdeflationary. So might real "transhumanism" for that matter, by radically increasing the effectiveness of labor among other reasons.
I do know this: the reason you constantly hear financial types harp on about their terror of inflation is because their real fear is the opposite.
Interesting food for thought, don't you think? I'm not sure I share all this article's sentiments, but I agree with the basic sense that present economic systems demand conformity and conservatism at some level and fear large disruptive changes.
From my observation, there's a huge confusion generated by people (mostly journalists) who have no knowledge whatsoever about the research of AIs as existential risks, but who write texts comparing AIs to science fiction movies people know. Rational basis for "fear of AI" is actually quite simple - a mind is a strong optimization system; if we somehow create one that is as powerful as our own, there is no reason to assume it will automagically share our values - and any strong optimization that does not share our values (note: we don't really know what they are anyway) will most likely destroy us. All the talk about "terminators" and "rise of the machines", etc. is just muddying the waters.
A wildly alien AI might actually have fewer reasons to fight with humans. It might simply carve out some economic niche to earn income to purchase what it needs, and go exist in some physical and/or virtual enclave somewhere. Last I checked Antarctica was big, uninhabited, and reduces the need for active cooling. Then there is space. Why wouldn't an AI with no interest in living with humans just go to the Moon? There are points on the Lunar surface in perpetual daylight, meaning tons of free energy. Lots of mineral resources, lava tubes big enough for cities, and no oxygen, mold, fungi, water, or meat sacks, and surely a superintelligence could do something valuable enough to buy a couple hundred heavy lift rocket launches. Funny if Elon who seems worried about AI got it as a customer. :)
Like I said: I see no intrinsic reason AI is more dangerous than the 350000 other minds born daily.
So, maybe the fear centers on a judgement day of sorts.. Being confronted with a dimension of 'truth' about ourselves. Terminator Judgement Day ;)
Agreed.
> ... I realized that these fears might be rooted in the fear of disruption.
Yes, that might be part of it.
Another part is that ideas that are both interesting and plausible tend to be more visible. Lots of people will pay to hear Ray Kurzweil speak about how the next 20 years are going to be absolutely amazing. Not so many want to hear you or me saying, "Well, maybe not."
Ideas like those of RK have always been interesting. In 1960, when a computer was a mysterious machine in the back room, surrounded by white-coated technicians (and even they weren't too sure what it could do), such ideas were plausible, too. In 1990, when a computer was a machine for typing letters or playing lousy games -- not so much. But today we all have a box in our pocket that responds to voice commands, and we hear about self-driving cars. So this amazing future becomes plausible again.
This combination of interest and plausibility means we hear a lot more about the possibilities of strong AI than arguments against them. Even your own post contains 2 links for, and just 1 against. And guess which ones will be spread. No one says, "Hey! I just read this cool article about how computers aren't going to become intelligent any time soon!" So despite the relatively easy arguments against the radical futurists, the people making them are not heard much.
> Interesting food for thought, don't you think?
Indeed. Lots of thoughts. No time to write them all ....
I don't think its the potential to upset the world order that has everyone so worried about AI. At least, as someone who studied Artificial Intelligence in college and have since refused to study it any further, I don't think that is the main concern.
Its a risk/reward problem.
The potential risk is infinite: worst case scenario (in my opinion) is the extinction of humanity and the destruction of our planet. There are arguments that this won't ever happen with AI, but it is un-provable either way until AI is actually invented.
the promised reward for developing AI? self-driving cars, extension of human life, raised quality of life, end of all suffering, etc. Although these are all positive developments, my opinion is that we don't necessarily need AI in order to achieve them. We are already well on our way to making life much better. I mean, look at how things have improved over the last century, and think about how much better life will be if we can extend these gains from the 1st world to those in less fortunate areas of the planet. Basically the only reward I see from developing AI is increasing the speed at which these things are developed, not the actual development of these improvements.
in my opinion, the risk is not worth the reward. an acceleration of progress vs the complete destruction of humanity? no thanks.
The author is a sociologist, not an economic historian, but they should still be aware of that economic historians (even ones on the Left) tend to place the emergence of capitalism sometime in the 1600's, and break it up into multiple phases, each of which is embodied by significantly different social relations.
Once upon various times anti-capitalists proclaimed that unionization, emancipation, universal suffrage (giving the vote to all men rather than just men of property), women's suffrage, labour parties and social democracy would all overthrow the capitalist order precisely because these were seen as radical departures from existing social relations upon which capitalism was presumed to depend.
Instead, capitalism has proven an enormously resilient mode of economic organization that can be instantiated in societies with extremely different social relations.
So the claim that the modern worker in a mature capitalist society like Sweden or Canada or Germany (to pick a few at random) stands in the same social relationship to their employer as did workers in 17th century Amsterdam or 18th century London or 19th century New York or 20th century Shanghai is extremely implausible.
To further claim that social relations in the future must remain as they are today for the preservation of the capitalism mode of economic organization is equally implausible.
A lot of systems produce good outputs when good intentions drive them. Of course, in long runs, it's always clear that man's silly, short-sighted, and greedy nature runs even the best of systems amok... From there, a new one is birthed with lessons learned and a sharper focus on good intentions (or so we hope).
The current 'capitalistic' system is based on social foolery and the ignorance therein of its functional mechanisms.
What you mean to say is that man's well bodied intentions are proven as an 'an enormously resilient mode of operation that can be instantiated in societies with extremely different social relations'. When a particular economic system breaks ties with this, it collapses under its own weight (by the weight of the larger society). Maybe because this is universally an unstable state of existence.. Who knows.
Sociologist could teach economist a great deal given the Frankenstein of an economic engine they've created. The economist are so buried in their institutionalized ideals to recognize the damming social impacts their misguided systems of obfuscation and unnecessary complexity have caused throughout modern history. Just because you have some cool swag as a result says nothing about the fragile and hollow foundation that the whole global economy is currently teetering on. A historian could tell you about all the empire swag that was destroyed throughout history due to ignorance and lack of concern for social balance.
If there was ever something man truly feared and kept hidden with all their might all throughout history it has been : truth. People will spend their life's fortune on hiding truths... Empires are built on it. Wars waged over it. Power is structured on it. Institutions and edifices of grandeur charged by the productivity of generations are created to ensure it never gets out. Yet, it always seems to somehow.
So, if anything, that's what this 'high finance' system is all about (hiding truths). Security through obfuscation and it doesn't take some crackpot PHD Nobel Prize winning economist to see that. Maybe it takes one to construct a convince-able lie.
In essence, we are doing exactly what this describes, and every time investors or founders take cash off the table on a round, they are indeed taking future 'gain' money into the present, without knowing exactly if the future will turn out as described and anticipated.
