Charles Holt Current Research
Citations
Charles A Holt - Google Scholar
28,980 citations, 1,151 new citations in 2024
H Index = 64, i10-index = 171
REPC in October 2024: United States (you rank 355 of 12811, top 3%)
Overview
Many of my coauthored papers develop the implications of bounded rationality models to evaluate human behavior in laboratory experiments (quantal response equilibrium). Adding noise to decision making does more than spread theoretical predictions around a central point, because of cascading cross-effects in games of strategy. The resulting QRE predictions typically explain dramatic differences between observed decision patterns and standard Nash equilibrium predictions that are based on perfectly rational best responses to even small payoff differences. Applications include attacker-defender and social dilemma games (the first two publications listed below). The quantal response equilibrium is a generalization of the commonly used Nash equilibrium that replaces best responses with better responses, and that has provided theoretical explanations of intuitive behavioral anomalies.
I also have used laboratory experiments to help design and implement large-scale auctions used in a range of public policy applications (an FCC combinatorial spectrum auction, ongoing emissions permit auctions for the Regional Greenhouse Gas Initiative, and reference price auctions for diverse groups of financial securities). Current projects include auctions for offshore wind-energy lease auctions in the US and the UK, and auctions for procurement of liquified carbon dioxide at industrial scale for the government of Sweden. My coauthored macro-finance experiments pertain to saving for retirement and capital investment in the shadow of price bubbles.
Finally, I have worked with coauthors to develop widely used measures of risk aversion and belief elicitation. The idea is to present people with a menu of simple row-by-row choices to identify a crossover indifference point that indicates risk preference (the Holt/Laury mechanism) or a subjective belief (the Holt/Smith Lottery Choice Menu). In addition, I have helped devise and test a new nonparametric “Directional Difference” test that is based on permutations of data differences in a setting where treatments have increasing levels of intensity.
Published in 2023
“Binary Conflict: An Experimental Study of Strategic Effectiveness and Equilibrium,” Charles Holt and Thomas Palfrey, pre-published online August 2023, in the American Journal of Political Science (a top three political science journal) https://doi.org/10.1111/ajps.12810
Abstract: Bilateral conflict involves an attacker with several alternative attack methods and a defender who can take various actions to better respond to different types of attack. These situations have wide applicability to political, legal, and economic disputes, but are particularly challenging to study empirically because the payoffs are unknown. Moreover, each party has an incentive to behave unpredictably, so theoretical predictions are stochastic. This paper reports results of an experiment where the details of the environment are tightly controlled. The results sharply contradict the Nash equilibrium predictions about how the two parties’ choice frequencies change in response to the relative effectiveness of alternative attack strategies. In contrast, nonparametric quantal response equilibrium predictions match the observed treatment effects. Estimation of the experimentally controlled payoff parameters across treatments accurately recovers the true values of those parameters with the logit quantal response equilibrium model but not with the Nash equilibrium model.
“Permutation Tests for Experimental Data” Charles A. Holt and Sean P. Sullivan, 2023, Experimental Economics, 26, 775-812.
Abstract: This paper surveys the use of nonparametric permutation tests for analyzing experimental data. This approach, which involves randomizing or permuting features of the observed data, is a flexible and convenient way to draw statistical inferences. It is particularly valuable when few independent observations are available, as is often the case for controlled experiments in economics and other social sciences. When viewed as a framework, the permutation method constitutes a comprehensive approach to statistical inference. In two-treatment testing, permutation concepts underlie popular rank-based tests, like the Wilcoxon and Mann-Whitney tests. But permutation reasoning is not limited to ordinal contexts. Analogous tests are easily constructed for the permutation of continuous measurements, and we argue that these non-ranked alternatives should often be preferred when working with continuous data. Permutation tests can also be used with multiple treatments, with ordered hypothesized effects, and with complex data structures, such as hypothesis testing in the presence of nuisance variables. Drawing examples from the experimental literature, this paper illustrates how permutation testing solves common data analysis challenges. Our aim is to help experimenters move beyond the handful of overused tests in play today, and to show how permutation testing constitutes a general framework for conducting statistical inference with experimental data.
“Endogenous Reference Price Auctions for a Diverse Set of Commodities: An Experimental Analysis” Olivier Armantier and Charles Holt, 2023 online preprint, Experimental Economics, 30:1-27, doi: 10.1007/s10683-022-09783-6
Abstract: This paper is concerned with multi-object, multi-unit auctions with a budget constrained auctioneer who has noisy value estimates for each object. We propose a new allocation mechanism, the endogenous reference price auction, with two key features. First, bids are normalized across objects using "reference prices." Second, reference prices are set endogenously using information extracted from the bids submitted. We report on an experiment showing that a simple endogenous process mitigates value inaccuracies and improves three performance measures: the seller's profit, allocative efficiency and total surplus. These results have important implications for large auctions used in practice.
Forthcoming in 2024
“Rent Dissipation and Streamlined Costs: Laboratory Experiments,” (forthcoming 2024) with Angela Smith, Games and Economic Behavior, preprint available online: https://www.sciencedirect.com/science/article/pii/S089982562300163X?utm_campaign=STMJ_AUTH_SERV_PUBLISHED&utm_medium=email&utm_acid=86810394&SIS_ID=&dgcid=STMJ_AUTH_SERV_PUBLISHED&CMX_ID=&utm_in=DM426533&utm_source=AC
Abstract: The effect of group size and effort cost on rent dissipation in the standard Tullock rent seeking model is examined using a laboratory experiment. Data support qualitative comparative static Nash equilibrium predictions as reductions in both group size and effort cost increase effort levels. However, effort and dissipation rates are above Nash predictions for all treatments. In addition, while cutting per-unit effort costs in half increases individual effort levels, they do not fully double as predicted and therefore lower rent dissipation, creating a policy-relevant “streamlining effect”. A quantal response equilibrium, which adds stochastic elements into the game structure, predicts the observed higher effort levels and dissipation but is proven to be unable to explain the streamlining effect. A modification of quantal response that incorporates implementation error noise and loss aversion outperforms other estimated models and tracks the observed all data patterns more closely, including the streamlining effect.
“Cost Curve Confusion: Teaching with Earnings Comparisons in Class Simulations,” (with Erica Sprott), forthcoming in the Journal of Economic Education, 2024.
Abstract: This paper explains how two Veconlab class “experiments” can be used to clarify common points of confusion about the cost curves (sunk, marginal, and average). In each case, the experiment can be motivated, framed, or explained with environmental policy applications that are provided in the “suggestions for discussion” section. The connections to climate change are used to enhance student interest, and the provision of relative earnings information during the experiment is an effective way to let students discover their own misconceptions.
“Quantal Response Equilibrium and Behavioral Game Theory” Charles A. Holt and Thomas R. Palfrey, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 2086 words.
Abstract: Quantal response equilibrium (QRE) is a statistical generalization of Nash equilibrium in which deterministic best reply functions are replaced by probabilistic better response functions. In this manner, choice probabilities are ordered by expected payoffs. As with Nash equilibrium, QRE is a rational-expectations fixed point: players’ beliefs about the probability distribution of other players’ actions coincide with the distribution of actions arising from probabilistic “quantal responses” to those beliefs. QRE provides a general unified framework for analyzing games in strategic or extensive form. Randomness in behavior is a feature of data from experiments with human subjects, and therefore QRE is useful in explaining intuitive behavior patterns that systematically deviate from Nash predictions in a wide range of social dilemmas, auctions, and voting games. In addition, QRE can be used as a structural foundation to estimate economically relevant behavioral parameters such as risk aversion and social preferences.
“Belief Elicitation” Charles A. Holt and Angela M. Smith, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1800 words.
Abstract Information regarding individuals’ subjective beliefs is useful in identifying learning behavior and evaluating theoretical predictions. Incentivizing belief elicitation is standard in experimental economics but variation exists in the procedure employed. The most common approach is the quadratic scoring rule (QSR). While QSR can be an effective framework, truthful revelation of beliefs is only optimal for risk-neutral subjects, creating potential bias that can be exacerbated by larger payoffs. Other elicitation methods, such as Becker-DeGroot-Marschak (BDM), only involve two possible payoffs and thus do not rely on assumptions of risk-neutrality. However, BDM is notoriously difficult for subjects to understand. A third lottery choice (LC) method maintains the incentive-compatibility of BDM but is presented in an easily comprehensible table of binary lottery choices between an event lottery and a lottery based on known probabilities. A comparison of these elicitation methods suggests that LC generates more accurate belief reporting than QSR and BDM.
“Experimental Analysis of Antitrust Issues” Charles A. Holt and Sean Sullivan, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1370 words.
Abstract: Controlled experiments provide unique perspectives on antitrust issues pertaining to behavior that is otherwise difficult to observe. One example is collusion, since the illegality of this conduct gives participants strong incentives to conceal it. Another example is predatory pricing, which may deter potential competitors without producing easily observable instances of exclusionary pricing, especially in the typical case of a multiproduct firm with evolving and uncertain cost conditions. This entry summarizes some of the insights that antitrust experiments have provided as well as the difficulties that have been encountered.
“Risk Preference Elicitation” Charles A. Holt, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1323 words.
Abstract: Economists have devised various menus for using survey respondents’ choices to infer risk preferences. A typical task offers a choice between a safe option (payoffs close together) and a risky option (more payoff spread). The Holt-Laury menu holds the payoff amounts constant and changes the probability of the high payoff, using a crossover point from safe to risky to infer risk aversion. The Eckel-Grossman menu holds the high-payoff probability constant (at ½) and presents subjects with a choice between alternative portfolios, with the payoff spread increasing for more risky options. A “bomb task” lets respondents choose both probability and payoff, with lower probabilities for options with higher payoffs. There is strong evidence that increases in payoff scale result in higher risk aversion. Women tend to be more risk averse than men, especially with “downside risk” (low probability of a low payoff) when the safe option involves no risk.
“Rent Seeking Experiments” Charles A. Holt and Angela M. Smith, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1150 words.
Abstract: Non-price competition, which can arise in situations where the price mechanism is impeded or bypassed, can lead to inefficient “rent-seeking” activities that are socially wasteful. One example occurs when firms incur costs to acquire a valuable monopoly right or capture other politically-created rents. Experiments are used to test the theory in an effort to extend knowledge gathered from existing empirical studies, which are limited due to lack of available data. The most common experimental structure involves implementing Tullock’s (1980) lottery-based allocation procedure, whereby an individual’s probability of winning a prize equals their chosen effort as a fraction of the sum of all efforts among group members. With risk-neutral individuals, theory predicts substantial rent dissipation that increases with group size, but surprisingly, is unaffected by changes in effort cost. While experimental evidence supports the predicted group size effect, rent dissipation is generally greater than predicted and increases with effort cost.
“Asset Market Experiments and Price Bubbles” Daniel Q. Harper and Charles A. Holt forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 826 words.
Abstract: The fundamental value of durable assets like common stocks is not known with precision, due to uncertainty about future dividends and changing circumstances. In contrast, it is possible to conduct a laboratory experiment for shares of an asset with a well-defined value that is determined by the present value of all dividends and any final redemption payments specified in the experiment. Price bubbles that rise above this known fundamental value are pervasive in laboratory experiments, especially in the presence of sufficient liquidity in the market to cover speculative purchases. Treatments that increase liquidity, impair subjects’ judgements, and restrict short selling have been shown to increase bubbles. Bubble activity is diminished when decisions are made by groups or when traders are selected on the basis of high cognitive ability.
Papers Under Review:
“Inventory Management with Carryover in a Laboratory Setting: Going Beyond the Newsvendor Paradigm,” Béatrice Boulu-Reshef and Charles A. Holt, December 2024, under review.
Abstract: This paper considers a stationary model of inventory management in a rich setting in which unsold units carry over, in contrast with the full depreciation of unsold units that is implemented in laboratory studies of the newsvendor problem. The model permits an array of costs associated with restocking, understocking, depreciation, financing, and holding inventories. The extra dimensions make it possible to hold the optimal inventory constant, while adjusting parameters that change the frequency of stockouts and the risk associated with storage and depreciation. This framework facilitates an investigation of factors that influence the nature and severity of behavioral biases observed in simpler newsvendor settings. Optimal inventory decisions are derived and tested with a laboratory experiment that is focused on the “pull-to-center” effect, the “recency” effect, the effect of increased up-front costs, and the effect of risk aversion.
“The Directional Difference Test” Charles A. Holt,, Daniel Kwiatkowski, and Sean P. Sullivan, initial draft, December 2023.
Abstract: When an experiment has more than two treatments that can be ranked in terms of intensity or predicted effect, it is natural to use a statistical test that has a directional alternative hypothesis. This paper uses Monte Carlo simulations to evaluate the power characteristics of the Directional Difference test, which has a test statistic that is the sum of treatment differences for all pairs of observations in different treatments. When sample data are distributed according to familiar distributions like the normal, uniform, and logistic distributions, the Directional Difference test is better able to detect true treatment effects than the commonly used Jonckheere-Terpstra test that is based on counts of “binary wins,” Experimenters wishing to better exploit the informational content of multiple-treatment samples should consider replacing the Jonckheere-Terpstra test with the Directional Difference test in appropriate circumstances.