Democracy: Global, Historical Measures Based on Observables

John Gerring @ UT Austin

Daniel Pemstein @ North Dakota State

Svend-Erik Skaaning @ Aarhus University

Daniel Weitzel @ Colorado State

Video Recording

Slides

Abstract: 

Most crossnational indices of democracy rely centrally on coder judgments, which are susceptible to bias and error, and also require expensive and time-consuming coding by experts. This project lays out an approach to measurement based on observables that does not sacrifice the nuanced quality of subjectively coded democracy indices.

First, we gather data for a wide range of observable outcomes that aim to capture different aspects of the democratic process. Next, we employ a random forest model in which an existing democracy index, Z, is the outcome and factual indicators, Xi, are the predictors. The model that provides the best fit to the outcome is understood as an alternate index, Zi, for that conceptualization of democracy.

Naturally, there is some information loss from Z to Zi. We show that the loss is minimal for indices that ascribe to an electoral model of democracy. Despite information loss, an index based on observables may be advantageous for some purposes. It is free of idiosyncratic coder errors arising from misinformation, slack, biases for or against a regime, or data entry mistakes. It is free of systematic bias that may arise from coders’ inferences about a country’s regime status, e.g., from its recent economic performance, episodes of civil unrest, public policies (right- or left-wing), alliances (e.g., with the West or against the West), and the time-period under review (e.g., historical or contemporary). The data collection procedure and mode of analysis is fully transparent and replicable, and the procedure is cheap to produce and easy to update. For any index, Z, an observable index, Zi, can be generated that offers coverage of all polities with sovereign or semisovereign status, surpassing the sample of any extant index. We show that expansive coverage makes a big difference to our understanding of some causal questions.

Bios: 

John Gerring (PhD, University of California at Berkeley, 1993) is Professor of Government at University of Texas at Austin, where his teaching and research centers on methodology and comparative politics. He is co-editor of Strategies for Social Inquiry, a book series at Cambridge University Press, and serves as co-PI of Varieties of Democracy (V-Dem) and the Global Leadership Project (GLP).

Dan Pemstein is a comparative political economist and methodologist who studies democratic institutions. Much of his current research examines challenges that digital networks pose to democracy and develops tools to better measure democratic institutions. He also has an ongoing research program that explores the interplay between legislative behavior, political careers, and party organization. He teaches courses on comparative politics, political economy, global public policy, and research methods.

Svend-Erik Skaaning is professor of political science at Aarhus University, Denmark. His research interests include the conceptualization and measurement of democracy, civil liberties, and the rule of law. His recent publications include the books Democracy (Reflections) and Democratic Stability in Times of Crisis.

Daniel Weitzel is a post-doctoral researcher at the University of Vienna and incoming assistant professor at Colorado State. His research interests focus on comparative politics and quantitative and computational methods.