Occasionally players are curious about whether their performance in the game is improving, or would like to compare their own performance with other players’, whether those others are current opponents, or celebrated names from earlier years of the game.
But it is not statistically sound to use the game’s standard points-awarded scheme to make such comparisons over time. That is because the number of players in a round has a strong effect on what the possible scores can be, and that means you can meaningfully compare a player’s score in the latest round with their score in a earlier round only on the condition that those rounds are matched as to number of participants, correct guesses and DQs.
The same is true of dealer scores. It is not hard to see that, say, D11 scores are not all alike. In Round 1858, the real definition collected 11 votes from 20 voting players, for a guessability of 52%. But in Round 3545, the real definition collected 11 votes from 14 players, for a guessability of 85%. Both dealers scored D11, but the two scores are not comparable because of the difference in the number of players.
To address this, we have adopted a rating system, that in principle is similar to the Elo system used in chess. In chess, a player's rating is a number that may change depending on the outcome of rated games played. After every game, the winning player takes points from the losing one. The difference between the ratings of the winner and loser determines the total number of points gained or lost after a game. If the higher-rated player wins, only a few rating points (or even a fraction of a rating point) will be taken from the lower-rated player; however, if the lower-rated player scores an upset win, many rating points will be transferred.
For Dixonary we could not use the Elo system directly. For a start, chess matches have only two players, and Dixonary rounds have around a dozen (and, in the past, several dozen). And, because some games take ratings very seriously, there have been improvements to the mathematics involved, in the years since the USCF adopted Elo (1960) and FIDE did likewise (1970). For our purposes, the most important of these improvements was taking multiple players into account. That resulted in the Plackett-Luce model, which Dixonary uses without modification.
In a game of (for example) 12 players, the rating model sets each of the eleven voting players individually against each of the ten others, and calculates a rating change for each pair. Then each player’s improvement is the sum of their rating changes in that round. Clearly, this process relies on computing support. Unlike Elo scores, working them out with a pencil and a pocket calculator is not an option.
By having ratings, we of course invite comparison with chess, and in the Elo system, chess ratings range between 1200 for club player to 2500+ for grandmaster. Dixonary ratings range between 10 for casual player to 36+ for grandmaster. We don't have any grandmasters. Dealers have a separate rating scale that also ranges up to 36.
This website publishes fresh ratings on the basis of the Scorekeeper’s 5-round rolling scores report, which appears shortly after the end of each round. The expected lag between rolling scores report and rating table is less than 24 hours.