Behind the Scenes: How Men and Women Respond to Failure

This project takes advantage of computers' superiority to humans in calculating. Chess computers (so-called "engines") can take a given chess position and iterate it forward by a given number of half-moves (so-called "search depth"). In chess, a move consists of White's turn and Black's turn, a half-move refers to each side's respective turns. This can quickly turn into several thousand potential half-moves for the next 5 moves in the game. 

So... how do you find the best move if you're White or Black?

How computers evaluate a position

For computers, the answer is a combination of algorithmic pruning and brute force. Through forward iteration, the computer quickly dismisses unpromising lines (i.e. pointless piece losses) and studies lines more deeply that preserve the pieces' current movement and coverage. The results of this search are summarized in the computer's evaluation of a position. 

The specific evaluation function that determines a position's evaluation varies with the chess engine used (I used Stockfish 14.2, the strongest chess engine at the time at depth 20). The computer evaluation that Stockfish returns is in centipawns, a value that approximately summarizes the value of a position in hundredth pawns. The metric is designed primarily for computers and as such, difficult to fully grasp as humans. The most common interpretation: A position evaluated at +300 centipawns for White is a position so advantageous that it's equivalent to White playing with three extra pawns (300 centipawns / 100 = 3 pawns).

From position to errors

Once I know how an engine evaluates a single position, the next step is to see how it measures the quality of a move or the mistake magnitude. This is where the project uses the centipawn loss metric. 

For every position, the engine does two things:

1. 1. It identifies the best move and evaluates the resulting position already accounting for future perfect play ((a) + (c)).

   2. It evaluates the position after the player’s actual move (b).

The difference between these two numbers is the centipawn loss of that move.

For example:

• The current position (a) is +120  (assuming the best move in (c) is played next).

• If you play a different move (e.g. (b)), the resulting position is +70.

The centipawn loss is therefore 50 centipawns (120-70), or 0.5 of a pawn worse than the best alternative.

Scaling Up: Millions of Moves 

Analyzing a single game is straightforward. But scaling this up to entire research projects requires substantial computing power. For example, in How Men and Women Respond to Failure: Evidence from Chess Tournaments, around 23 million moves were evaluated, a process that took about eight weeks of continuous computation. The Captured by Conflict project was even larger, requiring evaluations for 41 million moves, again nearly 16 weeks of nonstop engine work.

Before the engines can even begin, the positions must be converted into a format the computer understands. Chess games are typically stored in PGN (Portable Game Notation), which records moves in human-readable form such as:

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6

For engine evaluation, these moves first need to be converted into LAN (Long Algebraic Notation), where each move explicitly specifies the starting and ending squares. The same sequence becomes:

e2e4 e7e5 g1f3 b8c6 f1b5 a7a6

This transformation ensures that engines can parse and evaluate these millions of positions. Once the engine receives the position in this format, it outputs two key pieces of information which need to be extracted:

1. An evaluation score (e.g., +42 cp), representing the position’s strength in centipawns.

2. A best move suggestion (e.g., b5a4), which the engine considers optimal given its search depth constraint.

Both pieces of information must be extracted for every move in every game before further analysis can take place. To calculate centipawn loss, the engine suggestions also need to be swapped into the move string so to directly compare what was played with what the computer wanted.

Computer's Preferred Move: e2e4 e7e5 g1f3 b8c6 f1b5 a7a6 | b5a4 -> +42cp 

Player's Actual Move     : e2e4 e7e5 g1f3 b8c6 f1b5 a7a6 | b2c3 -> -250cp [Player lost a knight]

Up until now, this has all been string manipulation. Once we have the centipawn scores, we can calculate the difference and obtain the centipawn loss.

Centipawn Loss: +42cp - -250cp = 292cp [a significant blunder]