Strongest chess player, ever (opens in new tab)

> Stockfish might be able to tell you what it was doing, but not in a way that it would be reasonable for a human to follow.

I (a non-chess player) just tried to play a game against the highest level AI (and lost, obviously).

Doing an analysis of the game afterwards, this is exactly what I experience: I do "f4" and I'm told (through the analysis tool) that the best move was "Nf3".

Now, the obvious question this leads to is: why? Why was this a better move? I don't think that, as a human being, memorizing "best moves" is going to lead to much improvement: we need to know WHY that move was the best move.

I'm sure there is a human-friendly way to explain why one move is the best move, and why my move wasn't, but the computer probably doesn't know this explanation, because it's approaching it from a brute-force perspective.

Surely, a chess computer can brute force all possible combinations, and deduce that this was the best move. But when this is not possible for a human being, just informing the me that "what you just did was not the best move", doesn't really do much to help me (as an amateur player).

The game, for reference: http://en.lichess.org/ehWjHnIc#0

>Stockfish might be able to tell you what it was doing, but not in a way that it would be reasonable for a human to follow.

Looking at the code posted, if the sub-scores were stored in an array and only added at the end, it would be possible to compare the positions after two moves by sub-score and find the biggest differences between subscores. Then you could say that position A is better than position B because it avoid doubled pawns, or has better bishops, etc.

Chess is a fun hobby, and I feel it gets more fun the better I get. That's a good enough justification for me.

It's a place to start.

It's not a huge leap from modelling the game of chess to modelling the skill of chess-playing. Stockfish already offers post-game analysis. Imagine extending that to analyze a players entire career, and offer a series of problems, games against specially-constructed opponents, analysis of relevant historical games etc, all of which are designed to help the player improve. Given Siri, Google Now, Watson and so on, we're probably not far from being able to have a meaningful natural-language conversation with a computer on a narrow subject like chess.

One could imagine this kind of thing being extended to teaching other, similarly focussed skills, at least for beginners. Piano. Tennis. Rock climbing. Maybe even things like soccer and basketball.

But that kind of teaching-based-on-deep-analysis is a long way off for subjects like physics or electrical engineering. Computers can't do physics, much less evaluate human physicists. The best we can do in these areas is something like the Khan Academy, where computers present "content" created by humans, administer standardized tests designed by humans and present the results to humans for interpretation.

So yeah, teaching chess in a really sophisticated way isn't all that useful in the sense that physics or EE are useful. But really, if we could teach computers to understand physics better than people do, we'd use them to make breakthroughs in physics, and that would be a much bigger deal than being able to teach physics more effectively.

On the other hand, we don't play chess or tennis or piano because they're useful, so expert AI teachers for these subjects would be really valuable.

By far, the fastest way to improve at chess is with a coach. Books work in the beginning, but soon you are crawling around in the dark. You can't identify your weaknesses, so you can't correct them. After studying the wrong thing for a year, you fix one of your weaknesses by accident, and you improve. A coach bypasses all of that wasted time. The challenge is not to automate a chess curriculum. That already exists on many websites selling chess software. Those are useful to learn certain theory and burn it into your brain by drilling over and over. The challenge is to create a chess teacher that can identify your specific weaknesses and correct them. A middle ground approach might work well, where you take one mental model of chess and develop a program to train that specific mental model. For example there is the Nimzowitsch model where chess is seen as siege warfare, with specific meta-strategies. If that model fits with how you think, then great. But it doesn't fit everyone. One day this super efficient learning will be reality. It sounds kind of boring. With an exponential game like chess, everyone will be at about the same level, except a few who throw their life away chasing n+1 while everyone else settled for n and having a life.

An AI is going to be much cheaper and faster than books + practice + private lessons.

The game analysis is definitely the part of this that interests me the most. That it could beat me isn't that impressive—I'm not that good. But that it could tell me what I did wrong, so I could improve my game... that's really neat.

Yes that way also works!

But the xboard way allows me to modify the engine, get debug output, try various different weights for pieces, put two engines against each other to see how specific features affect game play...

It helps me demystify such sophisticated software.

Once you condition on the number of draws, you do get that binomial distribution.

Suppose you have a coin, which gives a random outcome X. But you can only observe the outcome of X when another independent binary random variable Y is true. How can you tell if X is biased? Since X and Y are independent, the observations where Y is false are irrelevant since they don't tell you anything about X. So you just keep the observations where Y is true, and from there you can apply a binomial statistical test to the observations of X.

[ In case you're wondering whether applying statistical tests to variable sample sizes is valid, the answer is yes: a p-value is a uniform random variable from the set of observables (augmented by a continuous random variable, since our set of observables is discrete) to [0,1]. Our p-value is a mixture of p-values on smaller sample sizes, so it is still uniform. ]

This is exactly what happens here: consider a random outcome {win,lose,draw}. If you don't have a draw, let Y be true and X be the outcome of the game. If you have a draw, let Y be false and X be a random coin with the same distribution as for non-drawn games. Then X and Y are independent random variables and the above applies.

Informally: draws are not useful information in determining whether there are more wins than losses.

The first analysis isn't terrible. It gets the important points right (that the draws are not evidence of difference in skill) and moves on to the remaining evidence (difference in wins to losses).

The draws are at best evidence towards equality (not against it). Allow them to vary and the likelihood of seeing a difference of 9 wins in 64 games with 45 draws moves up to 0.13 or 13% (when we assume the two players are identical, an appropriate null hypothesis) (even less significant). So in about one tournament in 8 you would expect this much of a lead, even if it was one algorithm playing itself. So from one tournament we say it is likely the one algorithm is in fact better, but it doesn't rise to the standard of being statistically significant.

<code>

# R code to empirically estimate two-sided probablity of

# seeing a lead of 9 games when 64 games are played

# and the assumed probability of a draw is 45/64

# with the null assumption win/loss odds are equal

simulate <- function(nplay,ndraw) {

   sample(c('w','d','l'),size=nplay,replace=TRUE,

      prob=c((nplay-ndraw)/2,ndraw,(nplay-ndraw)/2)/nplay)

}

wldiff <- function(v) { abs(sum(v=='w')-sum(v=='l')) }

set.seed(350920)

stats <- replicate(10000,wldiff(simulate(64,45)))

print(sum(stats>=13-6)/length(stats))

## [1] 0.1341

</code>

(it is weird that somebody, not me, created a throw-away account to make the original comment. likely they are involved in chess development, or know how quickly stat discussions go sideways)

That's pretty cool!

Can anyone inform an amateur like me what happens at 2:14 (https://www.youtube.com/watch?v=7YWYS209ydE#t=134) and before that as well. It seems to me that black captures his own pawn using his knight. Is that legal?

EDIT: Ah, I see. White pawn captures black pawn, and the knight captures the white pawn. I just happens so fast I didn't see the white move.

Wow, very cool!

Can someone explain what's going on at 20:24 : http://youtu.be/7YWYS209ydE?t=20m20s ?

He forks the king and the queen, then his opponent moved his queen!?! This is not a legal move right?

Rejecting an improvement because it's just "too much code" is... brilliant, disciplined, and extremely difficult to do. Such a high percentage of software development is maintenance, and it's easy to underestimate that long-term maintenance cost.

>sometimes proposed changes that increase the engine's strength are rejected because they're too much code

that seems insane.

Performance optimization, while it is not neglected, is not what separates chess engines. You don't win an exponential race with linear advantages.

> but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy.

Is computer assisted chess a thing? Perhaps with standardized hardware, but any software.

In chess I'm good at strategy but terrible at calculating and I miss obvious stuff all the time in my fight for strategy. I always thought I'd do great with computer assistance to look for the obvious stuff, and me telling the computer the long term strategy.

Funny commentary:

> That either shows a profound understanding... or a profound lack of understanding. ... ROOK B6! He has NO SHAME

Which chess engine is behind this app?

As far as I understand the app (whose avatar's age goes up to 23) actually plays better than the living world champion?

Stockfish has been available for quite a while, and over the last few years has been recognised as being the strongest open-source chess engine.

It's fantastic to see it take on the best commercial engines toe-to-toe and come out on top.

These ratings are computer ratings, and because they are playing very much isolates (computers vs other computers), without enough external human encounters, the ratings are consistent within the computer sphere, but not really like-for-like with human rating lists.

A lot has happened since Deep Blue - that was just a 6 game match, which really only proved that machines can play near a grandmaster level, but without nerves. And at times that is good enough to beat a world champion in match conditions, because the human is more susceptible to the psychological events.

Since Deeper Blue we've seen the emergence of chess engines on personal computers (rather than racks and racks of RS6000 mid-range servers), so Chess Genius, Fritz, Shredder, Junior, Rybka, Houdini and now Stockfish pushing chess engines ahead.

The Chess Genius - Junior levels show chess engines capable of matching grandmasters at blitz/active-play speeds, but still struggling at slower time controls.

Rybka onwards show a computer engine where elite grandmasters are seriously challenged.

Chess hasn't been solved. Just watching Magnus Carlsen's play shows we are expanding chess knowledge incrementally one game at a time. Yes, progress has slowed down, but the incremental improvement are still there.

Chess technique is refined to such a super high degree, but still far short of "solving chess". Perhaps this current generation can beat grandmasters when psychological factors are minimised, it's hard to tell.

It's certainly not as clear cut as a Porsche vs Usain Bolt over 100 meters.

The computer is incredibly strong and there is no interest for a Man again Computer match but I wouldn't say it's close to being solved or that it's impossible to make further progress for three reasons:

1) Computer vs Computer can still go both ways so clearly the one chess engines make mistake others can exploit; 2) Some sequences are still hard find for computers, e.g. [1]. The long king walk by White is beyond the horizon of the computer. 3) Computer + Man against Computer + Man correspondence matches do benefit from human intervention.

[1]: http://www.chessgames.com/perl/chessgame?gid=1124533&kpage=5

It's actually not that common. Players will use chess engines to analyze positions, but they will rarely play against them because it doesn't provide realistic practice for an actual chess match.

For example, it may be reasonable to play an aggressive variation against an opponent because you think they might have difficulty finding a response in time pressure, but a computer can make precise calculations in any situation, and so such a strategy almost always backfires.

What's more, chess is often abstracted at higher levels in terms of things like long term plans ("I want to put pressure on the c7 pawn and control c6") instead of concrete material gains, which allows players to make progress even in positions where there are no direct threats and no sensible exchanges of pieces. Computers when faced with such situations will know that the position is objectively drawn and so will just shuffle their pieces around aimlessly, and playing against this kind of thing is not very good practice for actual human opponents who will try to find ways to beat you anyway.

Only realistic if they were preparing for some sort of Man vs Computer tournament.

Computers don't play like humans, and they are consistent in their weaknesses as well as their strengths. As players are more exposed to playing against various chess programs, they learn what positions computers are not so good at (long-range strategical themes, beyond the typical move horizon). A local optimisation, of sorts.

So players start adopting an anti-computer style of play, which requires a different mind set to pull off. That in turn affects their natural style of play.

So I argue the opposite, for preparing for top-flight tournaments, players probably avoid / severely limit playing against chess engines, to avoid this natural anti-computer bias to their play.

Sure, the engines are good for checking variations and analysis, as an assistant. But not as a leader, nor as an opponent.

Olympic sprinters do not practice racing against race cars. Maybe for a publicity stunt. But not for improvement.

That's helpful, thanks.

In the 1800s, most chess was much more aggressive, and it was frowned upon to reject an offered exchange of pieces. Consequently, games were faster, moves were riskier, and everything was arguably more interesting.

In other words, modern humans have been playing in dull, more reliable ways for over a century, so even if not sore loserish, it's at least a bit like the pot calling the kettle black.

"Demolishes"? Hardly. It is so far tied with H4 on IPON. Even if it winds up on top, it will not be by a large margin.

I did. I was complaining about the rated strength of the AI levels, they're off by a wide margin.

Rybka is better than top-20, so I did not claim that.

I was a half-pro ~10 years ago. I won the championship of my country. I wrote an M.Sc. thesis on evaluation tuning. If I'd use the standard approach that I already know, 6 months is a conservative estimate. The question is - what for?