The risk of AI is that it will make humans uneconomic, in the same way horses became uneconomic and were by and large sent to the glue factories. The entirety of the homo sapiens experience, all the strategies we have in the modern era and all the reliable tactics we use to stay ahead of other lifeforms rely on us being much better at pattern identification than literally anything else. Once AI are just better at everything - creative activities, running militarys, economic decisions, legal decisions, etc - then it starts to come down to entirely a question of what robotics is capable of for how long humans can hold out. We can fight an army, but not economics.
I think an intermediate step in the process will be making some humans uneconomic.
There's a saying in Yellowstone - the reason bear proof garbage cans are so hard is that there is substantial overlap between the smartest bears and the dumbest humans.
If we're talking existential risk, this is like Rome v. Carthage. Humans aren't going to come back once we're a bit behind. Although I don't think there is much to do and humans are so dodgy that it isn't even obvious that we should resist if the opportunity comes to replace us with something that can be consistently rational and intelligent.
Which is to say, our intelligence is mostly tuned to increase the number of descendants we have.
If AIs are subject to evolutionary forces, the same will be true for them.
Given how powerful full automation is, owning that capacity is close enough to being equivalent to a state today, so private companies would become states basically by default (Workers? What workers? These robots just produce stuff for the shareholders); and also we could get anarcho-communism if normal people get to own their own personal Rossumovi Univerzální Roboti, which can happen very easily because all it takes is one person who thinks that's a good idea and log2(human population) self-replications of the RUR to give everyone their own personal robot.
https://www.math.stonybrook.edu/~azinger/research/IPD.pdf
Press and Dyson (2012) discovered a special set of strategies in two-player Iterated Prisoner’s Dilemma games, the zero-determinant (ZD) strategies. Surprisingly, a player using such strategies can unilaterally enforce a linear relation between the payoffs of the two players.
In particular, with a subclass of such strategies, the extortionate strategies, the former player obtains an advantageous share of the total payoff of the players, and the other player’s best response is to always cooperate, by doing which he maximizes the payoff of the extortioner as well. When an extortionate player faces a player who is not aware of the theory of ZD strategies and improves his own payoff by adaptively changing his strategy following some unknown dynamics, Press and Dyson conjecture that there always exist adapting paths for the latter leading to the maximum possible scores for both players. In this work we confirm their conjecture in a very strong sense, not just for extortionate strategies, but for all ZD strategies that impose positive correlations between the players’ payoffs. We show that not only the conjectured adapting paths always exist, but that actually every adapting path leads to the maximum possible scores, although some paths may not lead to the unconditional cooperation by the adapting player. This is true even in the rare cases where the setup of Press and Dyson is not directly applicable. Our result shows that ZD strategies are even more powerful than as pointed out by their discoverers. Given our result, the player using ZD strategies is assured that she will receive the maximum payoff attainable under the desired payoff relation she imposes, without knowing how the other player will evolve. This makes the use of ZD strategies even more desirable for sentient players.
https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Zero-dete...
That's a good point to make about AI. Despite all the mess about ChatGPT, for example, the way it writes answers is already above a lot of people. And those person can have jobs that aren't threatened by AI (by now), but in some cases we already have jobs that are in a battle against a superior opponent, without expectation of winning. It's a sad time to be below AI, and every moment we're getting below it.
I disagree.
It relies on our superior communication and cooperation skills. Our 'advanced pattern recognition', which, I think any study of the sheer amount of irrational behavior done by humans will show really isn't that advanced, are a side effect of that.
It basically explores all possible moves and all possible countermoves and selects the best one. Of course it is extremely well engineered and optimized, but this is the main concept.
As a consequence, it does not to be "trained" on a dataset with odds to play without a piece.
Removing a piece may mess up the values chosen for the piece square tables and certainly messes up with the opening book - but on the other hand, it makes it sooner for the engine to end up using it's tablebases in the end game!
Looks like it used to be brute force, as you say, but has since incorporated an NN after ita loss to AlphaZero
Nothing fancy. Just pure brute force.
The author is playing a 2-year old version (stockfish 14 vs 16 is current) on low depth (22 vs for the strength they use to get max rating I think they use at least 48 or so, which is clearly much stronger). If they played stockfish 16 on a decent depth it would be an annhilation I'm sure.
I'm a very average chess player, and there are moves I play that stockfish at depth 16 things are a blunder that at depth 32 it thinks are a brilliancy. These are typically things in openings I play a lot where I've had a chance to do some homework and learn the best moves on a high depth.
(Uh, I mean, could be done by a team which is able to train MuZero)
Life isn't a chess games either. It's not clear that a contest is happening or that only side will win. Humanity could, now, quickly stop AI. We could burn down all the data centers and chip fabs and kill everyone who knows linear algebra. But we don't because nobody ever sits down and says "Okay, the contest for existence starts now!"
Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.
A self-fulfilling prophecy that precipitates the very conflict it ostensibly avoids.
This is a very optimistic view. I'm not so sure that true AGI will be developed even in this century. GPT-4 and the likes are hitting a data limit, and are not close to being AGI.
Stockfish literally plays itself when finding the next move and because it's down a queen it just seeks further loss minimization and it won't assume the opponent will fall for tricky but unsound tactics.
A high rated human on the other hand will easily exploit my lower tactical ability and be tricky to get the advantage back. This applies especially on fast time control games.
To be able to do this, you would want to use a UCI-compliant chess GUI such as Cute Chess.[2] It lets you change the command line arguments you use for your engine, and you can play Human-vs-Human, Engine-vs-Engine, Engine-vs-Human, etc.
https://lczero.org/blog/2023/07/the-lc0-v0.30.0-wdl-rescale/...
They setup a lichess bot to allow you to play Leela at knight odds:
https://lczero.org/blog/2023/11/play-with-knight-odds-agains...
Chess can still be fun even if you don't have much theory onboard. You don't have to play the Grünfeld. Just play "theory-lite" openings like the London, and learn as you go.
What are some ways to improve at chess? Just go to lichess? Watch YouTube videos? I loved this article and the discussion of AI and would love to supercharge my learning if there are experimental ideas.
With three kids, my time is very scattered so I'm looking for ways to study chess in small fifteen minute chunks, as that is often all I have before falling asleep in one of their beds.
Bonus: are there ways to radically improve chess together (other than just playing, obviously!)
Has anyone used flashcards effectively for chess?
Then on chess.com you have courses and puzzles that help too.
You cannot really improve just by playing without learning theory.
I'm curious about how to teach theory to a ten year old. They are probably better at absorbing those things than I am, so maybe I'm not the best strategist anyway.
On the other hand two humans in the middle of nowhere can keep the species going indefinitely.
Software running on a piece of plastic and silicon is not going to win a war against humanity anytime soon.
Anthropomorphizing AGI is what leads to these silly thought experiments.
For the record, I don't personally believe LLM's, as they currently exist, could ever become AGI. But yeah, that's the popular thinking at least.
I'd say it's a far easier situation for the right AI than for humans. It's actually very hard for humans to think when they know bullets and missiles are flying in their direction.
Imagine a war against an opponent that never blunders. Nobody is every drunk, or sleeping, or fails to pay attention. Nobody is arguing about what's the right thing to do now. Nobody's running around like a headless chicken while artillery is falling around them. Nobody forgets to make use of the abilities of the equipment.
A war against an opponent that's always performing as well as it can would be quite the tricky scenario.
Of course a rogue AI with an easy to attack center would be enormously vulnerable, but I think such a thing shouldn't be assumed, since the reason to build AIs is to fight better, and any flaws in control and communications would be quickly exploited. So an AI controlled army is almost certain to be distributed to a large extent.
Actually this has already been studied in chess theory:
When I was ~8 my dad was teaching me to play. Not a single time he let me win, we could play like 100 games an evening (literally), he had incredible patience (sometimes I was taking lots of time to make a move, yet he never gave up on me)... It would have been more challenging to him (at least a little bit) if we knew the rule described in this article...
Not for the jobs threat. 'it took ouuurrr joubbbs' threat. This is already happening, and will continue.
But for the killer AI threat, the concept of 'material' in chess is good. As long as AI needs us for power, mining and manufacturing, we still have some advantage.
Until it gets to the point where it has so much power that it can threaten us to keep the power on, keep manufacturing chips, then we can still maneuver and win.
The whole problem with the movie "Colossus" is that they handed over nuclear launch control. So there was a threat of nuclear strike that forces humans to keep the power on. Without this, humans can come back, change direction.
Like with this excellent chess analogy, there will be a time when the AI is smarter, but humans will still have more material.
Lets say in 10-20 years we do get AGI, it is installed in F-16s. There will still be time, a gap period, where we can change direction. We see it has gone too far and we turn it off. The world realizes we are going to far and all countries come together to abandon AI.
The real threat is humans, threats of other countries, and profit motives. This is what will keep AI moving forward and in control. Because we'll be to scared or greedy to turn it off. NOT because of its superior strategic thinking. MOLOCH.
