Bias

Punished for Doing Well: Bias in Ranking Metrics at Swiss Tournaments -- British Parliamentary Debating, Team Points, and Speaker Points

Abstract

Motivated in part by our companion paper, we investigate bias in team points (wins, an relative/ordinal metric) and speaker points (an absolute/cardinal metric) at standard BP debating tournaments. We simulate, via the model of Barnes, Kehle, McKenny, and Lee (2020a) (with some modifications), tournaments with no bias in judging and find that ranking by team points is substantially biased against stronger teams and towards weaker teams, an original (to our knowledge) and significant result. We look at empirical data and calculate apparent bias in speaker points relative to team points against ESL and non-rep teams. We devise methods to account for bias in team points. After accounting for team point bias, we find essentially zero unaccounted bias in speaker points relative to team points against ESL and non-rep teams. We also find no statistically significant unaccounted bias in speaker points relative to team points against particular teams and individuals. We discuss how our work interacts with existing literature. 

Keywords: bias, speaker points, BP debating, ranking, Swiss tournaments