Code for generating the graphs can be found at https://github.com/kwojcicki/chess-stats
ACPL vs Rating
Chess, for practical purposes, is solved. Using online platforms players can compare their moves against optimal moves generated by chess engines/AI’s. Centipawn Loss (CPL) is a measure of how suboptimal one’s move is (in comparison to a chess engine’s move). A centipawn is equal to 1/100th of a pawn, meaning a CPL of 100 indicates the player made a move that lost the equivalent to 1 pawn. A CPL of 100 does not necessarily mean you lost a pawn unnecessarily; CPL also takes into account positions and potential mates. Average Centipawn Loss (ACPL) is the average CPL across a given game. ACPL is a positive number between 0 and infinity. An ACPL near 20 is excellent indicting near perfect play, whereas an ACPL above 300 either means you’re a 3 year old toddler or purposefully losing the game. Being able to calculate ACPL raises the question, is ACPL correlated to one’s rating?
In-depth CPL explanation
Let’s examine the following board
Both sides have similar pieces and similar positions. Stockfish gives this board a score of -0.69 (i.e. black is up 69/100th of a pawn or 69 Centipawn’s).
The optimal move is to move your bishop to b4.
That board is evaluated at -0.71.
If instead we played the bishop to c3, Stockfish will reevaluate the board to -0.89.
Since the optimal move resulted in a board evaluation of -0.71 and our move achieved an evaluation of -0.89, our move receives a CPL of -71 - -89 = 18.
The graph shows a negative correlation between rating and ACPL (ACPL decreases as rating increases) which follows common sense; as a player gets better they tend to perform moves that are closer to the optimal.
ACPL, as most performance indicators, require context. A player can achieve an ACPL of 20 in a variety of ways. They can immediately blunder their queen and attempt to stretch out the inevitable2 or they can play an excellent game.
Instead of looking at rating vs ACPL, we’ll look at rating vs move types. Online chess platforms tend to classify moves into several broad categories: best move, good move, inaccuracy (CPL between 50-100), mistake (CPL between 100-300) and blunder (CPL >200).
As one would expect, the number of blunders (game losing moves) drastically decreases with rating. It is unclear if the actual CPL per blunder decreases or increases with rating.
What is interesting is how mistakes stay relativity even, this may be due to the examined games being from the blitz variant of chess. Since the players have a limited time to think they are likely making moves that are good in the short term (3-5 moves ahead), but may be suboptimal in 10+ moves. Realistically, it will be difficult for humans to ever eliminate all mistakes and inaccuracies from their play, but concentrating on reducing one’s blunders even by 25% yields incredible gains in ones rating.
Conclusion
In both graphs, it is evident that the correlation factor between errors/ACPL and rating is low. Both are metrics that can be used to estimate the players’ rating in a range, but are unideal to accurately judge a player’s performance. Nonetheless, it is difficult to dispute the notion that better players have better ACPL.
Notes
[1] A sample of ~2500 games was taken using Lichess’s API, resulting in ~5000 ACPL samples being used. Players were pooled in buckets based on their ELO (i.e. the ACPL was averaged for players across 700-719, 720-739 etc). ↩
[1.5] There is a peculiar uptick in ACPL at the very upper ranks likely due to low sampling of 2900+ rated games. ↩
[2] One can imagine a scenario where the player immediately blunders their queen, causing their initial ACPL to be very high. As the game progresses, since the game is in such a dire position, all the player’s available moves will be roughly of the same quality, meaning any move the player makes will have a CPL close to 0. Eventually, as the number of moves grows larger the initial spike in CPL will be averaged off. ↩
Comments
Nice Article :D
Nice article :)
At 2800 plus, I’d suggest that the engine doesn’t actually understand the moves as well as the players, given the short time that lichess spends on the analysis. So at this level acpl loses it’s meaning, as there is no reference ‘perfect play’.
I recently read an article by chess.com (https://news.ycombinator.com/item?id=27188854) that seems to bolster that theory.
Also given that players at that level have a history of games between each other, I would imagine players will try to play moves that exploit their opponents weaknesses rather than always play optimally