Charles Holt Current Research
Citations
Charles A Holt - Google Scholar
27,660 citations, 1,235 new citations in 2023
H Index = 62, i10-index = 167
REPC in December 2023: United States (you rank 368 of 12811, top 3%)
Overview
Many of my coauthored papers develop the implications of bounded rationality models to evaluate human behavior in laboratory experiments (quantal response equilibrium). Adding noise to decision making does more than spread theoretical predictions around a central point, because of cascading cross-effects in games of strategy. The resulting QRE predictions typically explain dramatic differences between observed decision patterns and standard Nash equilibrium predictions that are based on perfectly rational best responses to even small payoff differences. Applications include attacker-defender and social dilemma games (the first two publications listed below). The quantal response equilibrium is a generalization of the commonly used Nash equilibrium that replaces best responses with better responses, and that has provided theoretical explanations of intuitive behavioral anomalies.
I also have used laboratory experiments to help design and implement large-scale auctions used in a range of public policy applications (an FCC combinatorial spectrum auction, ongoing emissions permit auctions for the Regional Greenhouse Gas Initiative, and reference price auctions for diverse groups of financial securities). Current projects include auctions for offshore wind-energy lease auctions in the US and the UK, and auctions for procurement of liquified carbon dioxide at industrial scale for the government of Sweden. My coauthored macro-finance experiments pertain to saving for retirement and capital investment in the shadow of price bubbles.
Finally, I have worked with coauthors to develop widely used measures of risk aversion and belief elicitation. The idea is to present people with a menu of simple row-by-row choices to identify a crossover indifference point that indicates risk preference (the Holt/Laury mechanism) or a subjective belief (the Holt/Smith Lottery Choice Menu). In addition, I have helped devise and test a new nonparametric “Directional Difference” test that is based on permutations of data differences in a setting where treatments have increasing levels of intensity.
Published in 2023
“Binary Conflict: An Experimental Study of Strategic Effectiveness and Equilibrium,” Charles Holt and Thomas Palfrey, pre-published online August 2023, in the American Journal of Political Science (a top three political science journal) https://doi.org/10.1111/ajps.12810
Abstract: Bilateral conflict involves an attacker with several alternative attack methods and a defender who can take various actions to better respond to different types of attack. These situations have wide applicability to political, legal, and economic disputes, but are particularly challenging to study empirically because the payoffs are unknown. Moreover, each party has an incentive to behave unpredictably, so theoretical predictions are stochastic. This paper reports results of an experiment where the details of the environment are tightly controlled. The results sharply contradict the Nash equilibrium predictions about how the two parties’ choice frequencies change in response to the relative effectiveness of alternative attack strategies. In contrast, nonparametric quantal response equilibrium predictions match the observed treatment effects. Estimation of the experimentally controlled payoff parameters across treatments accurately recovers the true values of those parameters with the logit quantal response equilibrium model but not with the Nash equilibrium model.
“Permutation Tests for Experimental Data” Charles A. Holt and Sean P. Sullivan, 2023, Experimental Economics, 26, 775-812.
Abstract: This paper surveys the use of nonparametric permutation tests for analyzing experimental data. This approach, which involves randomizing or permuting features of the observed data, is a flexible and convenient way to draw statistical inferences. It is particularly valuable when few independent observations are available, as is often the case for controlled experiments in economics and other social sciences. When viewed as a framework, the permutation method constitutes a comprehensive approach to statistical inference. In two-treatment testing, permutation concepts underlie popular rank-based tests, like the Wilcoxon and Mann-Whitney tests. But permutation reasoning is not limited to ordinal contexts. Analogous tests are easily constructed for the permutation of continuous measurements, and we argue that these non-ranked alternatives should often be preferred when working with continuous data. Permutation tests can also be used with multiple treatments, with ordered hypothesized effects, and with complex data structures, such as hypothesis testing in the presence of nuisance variables. Drawing examples from the experimental literature, this paper illustrates how permutation testing solves common data analysis challenges. Our aim is to help experimenters move beyond the handful of overused tests in play today, and to show how permutation testing constitutes a general framework for conducting statistical inference with experimental data.
“Endogenous Reference Price Auctions for a Diverse Set of Commodities: An Experimental Analysis” Olivier Armantier and Charles Holt, 2023 online preprint, Experimental Economics, 30:1-27, doi: 10.1007/s10683-022-09783-6
Abstract: This paper is concerned with multi-object, multi-unit auctions with a budget constrained auctioneer who has noisy value estimates for each object. We propose a new allocation mechanism, the endogenous reference price auction, with two key features. First, bids are normalized across objects using "reference prices." Second, reference prices are set endogenously using information extracted from the bids submitted. We report on an experiment showing that a simple endogenous process mitigates value inaccuracies and improves three performance measures: the seller's profit, allocative efficiency and total surplus. These results have important implications for large auctions used in practice.
Forthcoming in 2024
“Rent Dissipation and Streamlined Costs: Laboratory Experiments,” (forthcoming 2024) with Angela Smith, Games and Economic Behavior, preprint available online: https://www.sciencedirect.com/science/article/pii/S089982562300163X?utm_campaign=STMJ_AUTH_SERV_PUBLISHED&utm_medium=email&utm_acid=86810394&SIS_ID=&dgcid=STMJ_AUTH_SERV_PUBLISHED&CMX_ID=&utm_in=DM426533&utm_source=AC
Abstract: The effect of group size and effort cost on rent dissipation in the standard Tullock rent seeking model is examined using a laboratory experiment. Data support qualitative comparative static Nash equilibrium predictions as reductions in both group size and effort cost increase effort levels. However, effort and dissipation rates are above Nash predictions for all treatments. In addition, while cutting per-unit effort costs in half increases individual effort levels, they do not fully double as predicted and therefore lower rent dissipation, creating a policy-relevant “streamlining effect”. A quantal response equilibrium, which adds stochastic elements into the game structure, predicts the observed higher effort levels and dissipation but is proven to be unable to explain the streamlining effect. A modification of quantal response that incorporates implementation error noise and loss aversion outperforms other estimated models and tracks the observed all data patterns more closely, including the streamlining effect.
“Cost Curve Confusion: Teaching with Earnings Comparisons in Class Simulations,” (with Erica Sprott), forthcoming in the Journal of Economic Education, 2024.
Abstract: This paper explains how two Veconlab class “experiments” can be used to clarify common points of confusion about the cost curves (sunk, marginal, and average). In each case, the experiment can be motivated, framed, or explained with environmental policy applications that are provided in the “suggestions for discussion” section. The connections to climate change are used to enhance student interest, and the provision of relative earnings information during the experiment is an effective way to let students discover their own misconceptions.
“Quantal Response Equilibrium and Behavioral Game Theory” Charles A. Holt and Thomas R. Palfrey, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 2086 words.
Abstract: Quantal response equilibrium (QRE) is a statistical generalization of Nash equilibrium in which deterministic best reply functions are replaced by probabilistic better response functions. In this manner, choice probabilities are ordered by expected payoffs. As with Nash equilibrium, QRE is a rational-expectations fixed point: players’ beliefs about the probability distribution of other players’ actions coincide with the distribution of actions arising from probabilistic “quantal responses” to those beliefs. QRE provides a general unified framework for analyzing games in strategic or extensive form. Randomness in behavior is a feature of data from experiments with human subjects, and therefore QRE is useful in explaining intuitive behavior patterns that systematically deviate from Nash predictions in a wide range of social dilemmas, auctions, and voting games. In addition, QRE can be used as a structural foundation to estimate economically relevant behavioral parameters such as risk aversion and social preferences.
“Belief Elicitation” Charles A. Holt and Angela M. Smith, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1800 words.
Abstract Information regarding individuals’ subjective beliefs is useful in identifying learning behavior and evaluating theoretical predictions. Incentivizing belief elicitation is standard in experimental economics but variation exists in the procedure employed. The most common approach is the quadratic scoring rule (QSR). While QSR can be an effective framework, truthful revelation of beliefs is only optimal for risk-neutral subjects, creating potential bias that can be exacerbated by larger payoffs. Other elicitation methods, such as Becker-DeGroot-Marschak (BDM), only involve two possible payoffs and thus do not rely on assumptions of risk-neutrality. However, BDM is notoriously difficult for subjects to understand. A third lottery choice (LC) method maintains the incentive-compatibility of BDM but is presented in an easily comprehensible table of binary lottery choices between an event lottery and a lottery based on known probabilities. A comparison of these elicitation methods suggests that LC generates more accurate belief reporting than QSR and BDM.
“Experimental Analysis of Antitrust Issues” Charles A. Holt and Sean Sullivan, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1370 words.
Abstract: Controlled experiments provide unique perspectives on antitrust issues pertaining to behavior that is otherwise difficult to observe. One example is collusion, since the illegality of this conduct gives participants strong incentives to conceal it. Another example is predatory pricing, which may deter potential competitors without producing easily observable instances of exclusionary pricing, especially in the typical case of a multiproduct firm with evolving and uncertain cost conditions. This entry summarizes some of the insights that antitrust experiments have provided as well as the difficulties that have been encountered.
“Risk Preference Elicitation” Charles A. Holt, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1323 words.
Abstract: Economists have devised various menus for using survey respondents’ choices to infer risk preferences. A typical task offers a choice between a safe option (payoffs close together) and a risky option (more payoff spread). The Holt-Laury menu holds the payoff amounts constant and changes the probability of the high payoff, using a crossover point from safe to risky to infer risk aversion. The Eckel-Grossman menu holds the high-payoff probability constant (at ½) and presents subjects with a choice between alternative portfolios, with the payoff spread increasing for more risky options. A “bomb task” lets respondents choose both probability and payoff, with lower probabilities for options with higher payoffs. There is strong evidence that increases in payoff scale result in higher risk aversion. Women tend to be more risk averse than men, especially with “downside risk” (low probability of a low payoff) when the safe option involves no risk.
“Rent Seeking Experiments” Charles A. Holt and Angela M. Smith, forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 1150 words.
Abstract: Non-price competition, which can arise in situations where the price mechanism is impeded or bypassed, can lead to inefficient “rent-seeking” activities that are socially wasteful. One example occurs when firms incur costs to acquire a valuable monopoly right or capture other politically-created rents. Experiments are used to test the theory in an effort to extend knowledge gathered from existing empirical studies, which are limited due to lack of available data. The most common experimental structure involves implementing Tullock’s (1980) lottery-based allocation procedure, whereby an individual’s probability of winning a prize equals their chosen effort as a fraction of the sum of all efforts among group members. With risk-neutral individuals, theory predicts substantial rent dissipation that increases with group size, but surprisingly, is unaffected by changes in effort cost. While experimental evidence supports the predicted group size effect, rent dissipation is generally greater than predicted and increases with effort cost.
“Asset Market Experiments and Price Bubbles” Daniel Q. Harper and Charles A. Holt forthcoming 2024 in the Edward Elgar Encyclopedia of Behavioural and Experimental Economics, Swee-Hoon Chuah, Robert Hoffmann and Ananta Neelim, eds., 826 words.
Abstract: The fundamental value of durable assets like common stocks is not known with precision, due to uncertainty about future dividends and changing circumstances. In contrast, it is possible to conduct a laboratory experiment for shares of an asset with a well-defined value that is determined by the present value of all dividends and any final redemption payments specified in the experiment. Price bubbles that rise above this known fundamental value are pervasive in laboratory experiments, especially in the presence of sufficient liquidity in the market to cover speculative purchases. Treatments that increase liquidity, impair subjects’ judgements, and restrict short selling have been shown to increase bubbles. Bubble activity is diminished when decisions are made by groups or when traders are selected on the basis of high cognitive ability.
Papers Under Review:
Experimenting with Economics: A Behavioral Appetizer, Charles Holt and Erica Sprott, book manuscript, 235 pages with index, revised July 2023, under review with Princeton University Press.
Abstract: This book is intended to provide an experience-based introduction to economics, with a behavioral focus and an active-learning component based on classroom exercises. It may be used as a textbook in a stand-alone course, as a quick economics review for professional school students (MBA, Law, or Public Policy), or as an outline to be supplemented by more diverse readings and applications. The presentation is organized around a number of basic principles that underlie an economic way of thinking. These insights are so important that they should be reinforced by a process that allows students to discover the main ideas for themselves after participating in a classroom simulation or online “experiment.” The goal is go beyond documenting behavioral anomalies and help students navigate and anticipate them in a manner that promotes a deeper understanding and better decision making. Each chapter has an associated lab report that provides fill-in-the-blank structure for responses and the construction of figures. The lab reports are available in a format that permits easy copying and distribution during class, so that students can work together after the experiment is finished. The motivation is that students do not have to read a lot to learn a lot.
“Inventory Management with Carryover in a Laboratory Setting: Going Beyond the Newsvendor Paradigm,” Béatrice Boulu-Reshef and Charles A. Holt, December 2023, under review.
Abstract: This paper considers a stationary model of inventory management in a rich setting in which unsold units carry over, in contrast with the full depreciation of unsold units that is implemented in laboratory studies of the newsvendor problem. The model permits an array of costs associated with restocking, understocking, depreciation, financing, and holding inventories. The extra dimensions make it possible to hold the optimal inventory constant, while adjusting parameters that change the frequency of stockouts and the risk associated with storage and depreciation. This framework facilitates an investigation of factors that influence the nature and severity of behavioral biases observed in simpler newsvendor settings. Optimal inventory decisions are derived and tested with a laboratory experiment that is focused on the “pull-to-center” effect, the “recency” effect, the effect of increased up-front costs, and the effect of risk aversion.
Papers Being Revised:
“The Virtues of Lab Experiments” Gary Charness, James Cox, Catherine Eckel, Charles Holt, and Brian Jabarian, working paper being circulated for comments, December 2023, on the Top 10 List of Downloads in the SSRN PsychRN research design topic, January 2024.
Abstract: Physical lab experiments have played an instrumental role in sculpting the history of experimental economics, facilitating controlled information conditions, efficient monetary inducements, and exclusive advantages via immediate human interaction and engaging experiences. These unique benefits render in-person lab experiments essential for the future of experimental economics, complementing the growth of online experiments and the emerging AI revolution. We characterize the environments in which it seems particularly important to conduct lab-in-the-lab experiments. Overall, the lab benefits culminate in a comprehensive research procedure that produces precise and enlightening outcomes, ultimately enriching the domain of experimental economics, and potentially extending benefits to the broader realm of social science.
“The Directional Difference Test” Charles A. Holt,, Daniel Kwiatkowski, and Sean P. Sullivan, initial draft, December 2023.
Abstract: When an experiment has more than two treatments that can be ranked in terms of intensity or predicted effect, it is natural to use a statistical test that has a directional alternative hypothesis. This paper uses Monte Carlo simulations to evaluate the power characteristics of the Directional Difference test, which has a test statistic that is the sum of treatment differences for all pairs of observations in different treatments. When sample data are distributed according to familiar distributions like the normal, uniform, and logistic distributions, the Directional Difference test is better able to detect true treatment effects than the commonly used Jonckheere-Terpstra test that is based on counts of “binary wins,” Experimenters wishing to better exploit the informational content of multiple-treatment samples should consider replacing the Jonckheere-Terpstra test with the Directional Difference test in appropriate circumstances.
“Reverse Auctions to Procure Negative Emissions at Industrial Scale” Dallas Burtraw, Charles Holt, Åsa Löfgren, and William Shobe, Preliminary Draft: November 28, 2023, presented at the NBER Conference on Decarbonization.
Abstract: Many climate solutions including carbon dioxide removal (CDR) technologies require investments in capital intensive technologies that require large capacity investments and exhibit modest unit costs. Governments seeking to achieve net zero goals may invest directly in CDR to procure negative emissions credits to offset emissions in hard-to-abate sectors such as agriculture. In a procurement auction for a declining cost industry, the optimal allocation will generally require all winning bidders operating at full capacity, but this may not fit within the government’s budget, leaving one or more winning bidders at the margin, operating at less than full capacity, and with higher average costs. Protection can be provided to the marginal bidder by letting bids specify a range of acceptable quantities up to full capacity. The auction can be executed with sealed bids (with minimum quantities) or by having the proposed bid price be lowered sequentially in a “clock auction” with quantity intervals specified by bidders at the current clock price. We consider the performance of sealed bid and clock auctions, in the presence of 1) a fixed government procurement budget, 2) “common value” uncertainty about the true per-unit production cost, and 3) the presence of a large fixed cost. Laboratory experiment simulations with financially motivated human subjects are valuable for testing and developing auction designs that have never been used before, without relying on theoretical properties that depend on implausible assumptions of perfect cost information and “truthful bidding.” Preliminary experiment results indicate that winner’s curse effects (bidder losses) are infrequent in both auction formats (clock and sealed bid), but the clock tends to restrict bidder profits in a manner that reduces the average cost for the buyer of the “units” representing CDR. Our experiments are informed by the projected use of auctions by the government of Sweden to procure carbon capture and sequestration from its domestic energy and wood products industries.
A Behavioral Study Life-Cycle Consumption and Retirement Savings: Is Roth Better? Clement Bohr, Charles Holt, Alexia Schubert. We plan to revise and add a high-tax treatment in spring 2024 before submitting to a journal.
Abstract: This paper considers the choice between Traditional (tax-deferred) and Roth (tax-prepaid) retirement accounts. A laboratory experiment avoids selection effects that complicate analysis of naturally occurring savings. Subjects receive exogenous incomes in working periods, followed by “retirement” periods with no income. Subjects decide when to convert lab dollar accounts into “take-home pay,” with diminishing returns, which determines retirement savings and tax payments. The experiment is motivated by a perception that Roth accounts are better for most people, although the flat profiles of income, tax, and interest rates are implemented to eliminate those advantages. We focus instead on behavioral factors like an avoidance of present taxes by saving more with a tax-deferred IRA, even though optimal consumption and after-tax savings are identical for both treatments. Observed savings are suboptimal in both treatments and are influenced by gender, patience, and risk aversion measures. Post-retirement consumption is higher in the traditional IRA treatment.
Information Integration in Experimental Asset Markets with Insiders and Durable Assets, Daniel Q. Harper, Charles A. Holt, and Margaret M. Isaacson, being revised for submission in January 2024.
Abstract: This study tests the ability of asset markets to integrate and disseminate private information about durable assets. Subjects trade dividend-paying durable assets for 15 periods, after which the assets are redeemed for a value randomly drawn from a known distribution. Some subjects are “insiders” who know with certainty the redemption value of the assets. The number of insiders is varied across sessions. Rational expectations models predict that insider information is immediately incorporated into the market and no price bubbles occur, regardless of the number of insiders. Preliminary findings suggest that markets with insiders experience price bubbles, and that bubbles are larger in markets with less insiders.
Tobin’s Q, Liquidity, and Speculation in Laboratory Markets, Daniel K. Harper and Charles A. Holt, revised November 2023.
Abstract: This paper uses a laboratory experiment to study channels through which excess liquidity and Tobin’s q affect capital investment. Subjects produce and trade capital goods in a multiperiod market in which the capital stock depreciates and is subject to convex production costs. Treatments vary the marginal cost of investment, the aggregate cash level, and the relative cash endowments of individual subjects, holding aggregate cash constant. Increasing aggregate cash increases price bubbles for capital goods. In contrast to q-theory predictions of no effect, there is excess production of capital goods relative to a first-best optimum in markets with high cash endowments.
“An Experimental Analysis of the Shanghai License Plate Auction: Revenue and Efficiency Reduction” Evan Zuofu Liao, Charles A. Holt, and Penny Huixin Yang,
revised December 2023.
Abstract: Monthly auctions of vehicle licenses in Shanghai were reconfigured in 2008 in an attempt to reduce prices by giving bidders the option of modifying initial bids. The modified bids are required to be within a narrow band around the lowest accepted bid at that point. The bidding constraints limit price competition, and therefore, allocation efficiency may be degraded if intended revenue reductions are achieved. The effectiveness of the new procedures, however, is unclear in light of sharp subsequent increases in license prices, even after the license quota was doubled. This paper reports a laboratory experiment designed to evaluate the revenue and efficiency consequences of the revised Shanghai auction format, in comparison with the previous sealed-bid procedure, or with a lottery-based allocation used in other cities.
“The Winner’s Curse in Offshore Wind Auctions: An Experimental Comparison,” (with Erica Sprott), presented at the Southern Economics Association Meeting, Ft. Lauderdale, November 2022.
Abstract: The Biden Administration’s recently announced sale of new offshore wind leases is intended to address U.S. climate goals of expanding renewable energy in the next decade. The United States currently employs a simultaneous ascending clock auction to allocate offshore wind leases to prospective electricity generators. In contrast, the United Kingdom uses a multi-phase design including (1) a multi-round discriminatory auction to allocate the leases and (2) a ‘contract for difference’ reverse auction to guarantee revenue per megawatt hour of electricity produced. The United Kingdom’s design is advertised as decreasing uncertainty for clean energy producers in an industry susceptible to the winner’s curse. Contrary to the intention of the design, we find in a series of experiments that the U.K. auction mechanism is more susceptible to the winner’s curse
despite no significant differences in auction revenue. Beyond an experimental setting, these results have implications for improving offshore wind lease auction design and encouraging growth in the renewable energy industry.
“Can Competition Promote Trusting Behavior? Reciprocity, and Pro-Social Behavior in Repeated Interactions,” Charles A. Holt and Erica R. Sprott, presented at the North American Economic Science Association Meetings, October 2023.
Abstract: This paper uses a laboratory experiment to study trust and trustworthiness in dynamic games in which pairings are determined by mutual consent of investors and entrepreneurs. Profitable relationships can be scaled up, and unprofitable relationships can be terminated by the withdrawal of one of the parties. Subjects in the experiment are forced to choose which pairings to scale up and which ones to deactivate, and the competition to be included in mutually profitable pairings tends to produce sustained, high levels of pro-social behavior.
“The Allais Paradox and Bipolar Risk Preference: An Analysis with Chernoff Faces” (with Irene Comeig) initial draft presented at the North American Economic Science Association Meetings, October 2023.
Abstract: Subjects in a laboratory experiment are presented with a small set of paired lottery choices that can be used to analyze two related choice patterns: 1) Allais Paradox violations of expected utility, and 2) a switch from avoidance of downside risk (small probability of a relatively small payoff) to a preference for upside risk (small probability of a relatively large payoff). A large majority of financially motivated subjects exhibit one or both of these patterns. In contrast, only a minority of subjects exhibit risk aversion in all choices or risk neutrality/seeking in all choices. Applications for teaching in a behavioral economics class are also discussed. The observed decisions and demographics for each subject can be represented as a stylized facial expression (a Chernoff face), before being analyzed through the lens of prospect theory during a class discussion.
“Teaching Average vs Marginal Analysis with Green Eggs and Ham,” (with Madeleine Green and Juliette Sellgren), initial draft presented at the North American Economic Science Association Meetings, October 2023.
Abstract: An understanding of the difference between average and marginal effects is an essential element of effective decision making. Yet economists find that consumers often misperceive incremental gains or costs in all but the simplest environments. This paper presents a class experiment based on incremental production decisions with differing marginal products, with the intent of introducing students to marginal thinking and discouraging biases that result from comparisons of averages. The structures of the production functions for alternative products are not explicitly provided, and are revealed as students make labor input allocation choices. Earnings comparisons between students in the class, both during and after the exercise, are used to demonstrate the earnings improvements associated with marginal decision-making. The data from a parallel set of experiments done with financially motivated subjects is used to develop an attraction-reinforcement learning model that tracks adjustment patterns for a significant subset of the subjects. Another treatment, in both teaching and research sessions, involves chat between pairs of students or subject. Related psychological theories, including melioration, are discussed.