Evolutionary game theory studies the behavior of large populations of agents who repeatedly engage in strategicinteractions. Changes in behavior in these populations are driven either by natural selection via differences in birth and death rates, or by theapplication of myopic decision rules by individual agents.
EVOLUTIONARY STABLE SRATEGY
In a symmetric normal form game, an evolutionarily stable strategy is a (possibly mixed) strategy with the following property: a population in which all members play this strategy is resistant to invasion by a small group of mutants who play an alternative mixed strategy.
APPLICATIONS
The Logic of Animal Conflict
In their seminal paper, Maynard Smith and Price made an attempt to explain why, in many biological populations, aggression is much less common than aggressive displays . Such a behavioral strategy, initially called as a retaliator, was thought to be evolutionary advantageous in many situations, as it rarely led to lethal injuries. Considerations of trade-offs between different strategies in terms of their impact on the population survival have became the focus of many models studying animal contests developed since then . One of the best-known games describing an animal contest is the Hawk–Dove game, in which individual incentives have a similar structure to another well-known game, the Snowdrift game . Here, Hawks exhibit aggressive behavior and fight for the resource that increases their fitness by V. If fought back, they suffer injuries that can reduce their fitness by some cost C. On the other hand, Doves tend to play nicely and share resources equally unless they encounter a Hawk, in which case they flee and give up the resource. Such a simple setup provided a lot of fruitful insights into many questions, including mate choice, evolution of personalities , cooperation , social structure , and evolution of aggression itself . However, even though the game only involves two parameters, V and C, estimating their values for making directional predictions in evolutionary dynamics is often a challenging task in practice. Within our special issue, Galanthay et al. suggest a consumer-resource model that offers additional insights into the evolution of aggression. Their setup allows to study optimal aggression levels as a function of ecological and evolutionary parameters, such as the richness of the environment, animal mortality, and the amount of time spent fighting.
Of course, not all animal interactions resemble a conflict. Often, species find ways to coexist for the benefit of each other, and it is believed that such an ability gave rise to multicellularity . Even if driven by purely selfish incentives, organisms may still find it beneficial to share resources with others as long as their own needs are satisfied. Mutualism is one manifestation of this principle often referred to as pseudo-reciprocity: two species interact and, while both might have to suffer some cost, they also both benefit from the interaction . While ubiquitous, stability properties of mutualistic interactions are not entirely clear , which is addressed by Gokhale et al. in this special issue. The authors introduce a new approach that incorporates within-species interactions and demonstrates that mutualisms can be stable across various environmental conditions without altering the parameters related to between-species interactions. Their study emphasizes the importance of balancing both within- and between-species interactions in theoretical modeling to enable the persistence of mutualisms even in the face of ecological disruptions. This framework aligns with emerging empirical evidence highlighting the role of community-level dynamics and population interactions in sustaining mutualistic relationships.
Evolutionary Games and Health
Evolutionary games were not only applied to animal kingdom, but to a much wider spectrum of biological taxa, for example, microbes and diseases. If we assume that a disease is subject to Darwinian evolution, then evolutionary game theory is a very good and perhaps the best way to frame and study it . In case of cancer, for example, the normal form of the game would include cancer cells as the players, their heritable traits as their strategies, and their survival and proliferation (fitness) as payoffs . First papers on evolutionary game theory of cancer were published in 1990s . Since then, over 120 publications on cancer have called their research explicitly game-theoretic. Game theory has provided valuable insights in cancer evolution and treatment . When treatment resistance evolves as a quantitative trait, a natural way to model cancer under treatment is Darwinian dynamics . Game-theoretic reasoning has led to the development of evolutionary therapies (also known as adaptive therapies), which aim at anticipating and forestalling treatment resistance in advanced cancers and outperform standard of care in initial clinical trials . Better game-theoretic models will likely lead to better understanding of cancer and subsequently to better cancer therapies .
In this special issue, Bayer and West contribute to such a better understanding. They utilize evolutionary game-theoretic models of cancer under treatment. More specifically, they consider an evolutionary game with two phenotypes of cancer cells. Treatment reduces the growth parameters of the fitness matrix proportionally to the dosage. Subsequently, Bayer and West explore the link between frequency-dependent competition of cancer cell phenotypes and the ‘treatment convexity’ of cancer. Treatment convexity is the measure of the differences of the patient’s response to treatment schedules with identical cumulative dose levels but different dose variances. Their models of cancer growth include two cancer phenotypes and are based on the ‘gains of switching’ literature. The games they study belong to the following four classes: prisoners’ dilemma, coordination, anti-coordination, and harmony. They observe that, as long as there is no switch in a game class, the equilibrium growth rate is a linear function of the dose for all considered classes, except for anti-coordination games. A switch between game classes due to treatment leads to a wide variety of treatment convexity outcomes. Bayer and West’s work partially explains recent findings in the oncology literature, where such switches between game classes due to treatment were observed .
Transmissible diseases, such as transmissible cancers or Covid-19 , can be modeled and analyzed with tools from evolutionary game theory, too . One can take a microscopic perspective and focus on disease evolution within an individual, or focus on interactions among humans and human behavior in general and their impact on the disease spread , as also analyzed by Hota et al. within our special issue. They introduce a dynamic population game model to study the behavior of a large population during an infectious disease or epidemic, where individuals have five possible infection states and make choices regarding vaccination, testing, and social activity. Hota et al. analyze the evolution of infection states and individuals’ behavior, finding stationary Nash equilibria and exploring transient disease dynamics through evolutionary learning. Moreover, the proposed framework allows for the application of evolutionary learning strategies and exploration of the joint evolution of infection states and players’ decisions. Their results demonstrate a difficulty for an individual to decide between vaccination, testing, and social activity under varying conditions.
A possible extension of Hota et al’s work is to include a mediator, whose main goal is to steer the system into a desired direction. This can be done by adding a Stackelberg leader to the game , which would allow to focus on finding the best strategies for minimizing the disease spread. In general, games between a rational leader and evolutionary followers termed Stackelberg evolutionary games (SEG), such as those discussed by Kleshnina et al. within this special issue, can frame many different application domains, such as in fisheries management , pest management , managing antibiotic resistance, and conservation ecology. Here, the followers’ eco-evolutionary response is modeled through Darwinian dynamics . Kleshnina et al. highlight mathematical challenges associated with extensions of SEG theory to include vector-valued management strategies and vector-valued traits in the evolving species, and traits influencing different life-history stages of the species under management. Such extensions would allow for further expansion of SEG applications by capturing their key complexities. However, fundamental theoretical results, including stability and reachability of the Stackelberg and Nash equilibria, are necessary to be derived first. To accomplish this, the authors encourage the participation of mathematicians from diverse disciplines.
Evolution of Cooperation
Another prominent application of evolutionary game theory is the evolution of cooperation. From bacterial biofilms to human societies, actions that benefit others against a cost to the helping individual are ubiquitous. Despite its vulnerability to exploitation, cooperation persists and flourishes in many species and the questions of why and how it happens became one of the main focuses in the field . One of the most inspiring examples of where the persistence of cooperation is surprising is social dilemmas . Here, interacting individuals can choose an action that benefits their partner against a cost to themselves. In its most classical form, payoffs are such that cooperation is socially optimal, yet defection is the individually rational choice. Such a payoff structure creates tensions between the group as a whole entity and each individual member, making the evolution of cooperation puzzling. Many potential mechanisms for cooperation have been suggested, such as reciprocity, punishment, relatedness, network structure, and many more .
When modeling the evolution of cooperation, two-player two-action games like the prisoner’s dilemma , snowdrift or stag hunt became the go-to modeling choices. Depending on the incentive structure and exact modeling scenario, either of these games can be used for studying cooperative behavior . However, before choosing the exact model, one has to define what does it mean to cooperate or to defect in mathematical terms. In this special issue, Peña and Nöldeke argue that despite the richness of literature on the topic, there is no clear-cut mathematical definition of cooperation. The authors also point out that extending the model to multi-player interactions adds more technical complications. Peña and Nöldeke suggest a unifying approach to multi-player two-action games of full information. Their approach ensures consistent definitions of cooperation and cooperative dilemmas. By exploring the evolutionary equilibrium structure, they show that prisoner’s dilemma and snowdrift games feature exclusively inefficient equilibria, while stag hunt games might exhibit more cooperation than expected. In addition, they identify conditions for when full cooperation is socially optimal.
One potential mechanism for sustaining cooperation is partner choice, where group members may be able to choose with whom they would like to interact . It was shown that such an assumption may promote cooperative behavior . Within our special issue, Martin and Lessard analyze a game where group founders may express preferences for the group composition resulting in assortment. Here, individuals engage in a two-player prisoner’s dilemma with two strategies in both infinite and finite populations. The authors show that if the group founders have stronger preferences for more homogeneous groups, then cooperation is more likely to evolve and be promoted independent of the population size under certain conditions. The first condition is referred to as ‘global selection,’ where individuals contribute proportionally to their average payoffs. The second condition is referred to as ‘local selection’; here, the individuals contribute equally and cooperation has to be risk-dominant over defection in the absence of assortment. They also consider stochastic variability in the assortment level and/or the group size.
Direct and Indirect Reciprocity
Among the different mechanisms for cooperation, reciprocity has received particular attention, starting with the foundational papers by Trivers and by Axelrod and Hamilton . This mechanism captures the idea that individuals have more of an incentive to cooperate if their prosocial actions now increase the chance to benefit from others’ cooperation in future. The literature distinguishes several forms of how reciprocal cooperation might unfold. Perhaps the most prominent form, direct reciprocity, is based on mutually cooperative exchanges in fixed pairs, or in small groups . Here individuals engage in a repeated game for several rounds. This allows players to adopt conditional strategies, such that they are more likely to cooperate with another cooperator. Prominent strategies of direct reciprocity are Tit-for-Tat , Generous Tit-for-Tat , or Win-Stay Lose-Shift . A different form of reciprocal cooperation is described by the literature on indirect reciprocity . Here, players no longer interact in small and stable groups but they rather interact in large populations. Cooperation is maintained by social norms . Cooperative population members earn a positive reputation, which in turn makes it more likely to receive future cooperation. Prominent norms for maintaining cooperation are Image Scoring , Generous Scoring and the norms of the ‘leading-eight’ .
Within our special issue, the paper by Podder and Righi explores the effect of reciprocity in a more complex environment than typically studied. Rather than only allowing for cooperation and defection, they also allow ‘loners’ who abstain from the collective action . From the viewpoint of indirect reciprocity, this added possibility raises interesting questions. For example, what kind of reputations should be assigned to loners, compared to defectors? Once reputations are assigned, how should people decide whether to cooperate, given the reputations of other group members? Podder and Righi use simulations based on a genetic algorithm to address these questions. Exploring different group sizes and different social norms, they find that cooperation is most likely to evolve when a moral system is in place that assigns strictly worse reputations to defectors than to loners. But even then, the effectiveness of indirect reciprocity to maintain cooperation in group interactions is limited to comparably small groups. For group sizes beyond ten, individuals are predicted to abstain from collective action altogether.
Social Norms and Institutions
A different mechanism for cooperation, particularly relevant for humans, is the use of incentives, such as punishment or rewards . This mechanism has received particular attention after the seminal behavioral experiments by Fehr and Gächter . They showed that once people can punish each other, groups immediately become more cooperative, often rendering any explicit punishment needless. Since then, researchers have explored in which societies punishment is effective , whether it helps to increase overall welfare , and whether rewards or punishment are more favorable to the evolution of cooperation . From a theoretical viewpoint, incentives lead to a shift of the problem. Instead of explaining why people cooperate, corresponding models now need to explain why individuals are willing to pay costs to reward or punish each other, leading to a so-called second-order dilemma .
In addition, another problem is to explore how incentives should be used optimally, to make it most likely for cooperation to evolve. This is the problem that Cimpenanu, Santos, and Han explore. They consider a model in which individuals populate a heterogeneous social network. In general, population structure and spatial games have been shown to have a considerable impact on evolutionary dynamics , and on the emergence of cooperation more specifically . In Cimpenanu, Santos, and Han’s paper however, there is also an exogenous social planner who additionally seeks to promote cooperation. To this end, the social planner decides how to administer rewards, depending on how abundant cooperators are, and depending on the position of a cooperator within the network. The authors find that rewards can sometimes be counter-productive. Depending on how individuals update their strategies, on the network structure, and on how rewards are administered, rewards sometimes reduce overall cooperation. This work thus serves as an example that well-intended interventions can backfire if a population’s social dynamics are not taken into account.
Evolutionary Dynamics and Learning
If researchers are to describe the dynamics of an evolutionary game, they first need to determine by which process strategies change over time. By now, the literature knows of a number of different processes. For example, birth-death processes assume that individuals with low payoff (fitness) are more likely to die, and/or that individuals with high payoffs are more likely to reproduce . In contrast, a pairwise-comparison process is more adequate when describing the change of strategies due to social learning. In addition, the literature considers several other evolutionary dynamics, such as best-response dynamics, logit-response dynamics , fictitious play , and many others . While all of these models make plausible assumptions on how individuals revise their strategies, they often lead to subtle differences in the resulting dynamics.
In this special issue, Couto and Pal describe the properties of introspection dynamics, a process that is particularly suited to describe decision-making in asymmetric games . This process assumes that at regular time intervals, a random group member is given an opportunity to revise their strategy. This player then compares their current payoff to the payoff the player could have obtained by playing a randomly selected alternative strategy. The higher the hypothetical payoff of the alternative, the more likely the player is to switch. This elementary updating procedure results in a stochastic process on the space of all action profiles. While this process is relatively well understood for two-player games , Couto and Pal provide a general formula for the invariant distribution of this process for arbitrary multi-player games. In several special cases, including additive games, potential games, and symmetric games with two actions, this invariant distribution takes a particularly simple form, which the authors rigorously characterize. In addition, they apply their results to a number of instructive examples, such as the public goods game.
Evolution of Preferences
The introspection dynamics discussed by Couto and Pal can be seen as one example of a strategy adoption process during the lifetime of an organism. In parallel to biological studies, economists expanded the application of evolutionary game-theoretic reasoning to human behavior to a different level. By interpreting the evolutionary process directly as an inheritance process, they assumed that individuals are born with preferences over strategic choices and these preferences dictate economic choices during the lifetime of an individual . In this interpretation, selection acts at the level of preferences, which is often referred to as an indirect evolutionary approach . One key difference from the methods adopted in biology is that individuals are equipped with a utility function, which may include elements other than the direct payoff from the interaction. Some of the most well-known utility functions were formulated to explain abundance of altruism or spite , and morality . These studies demonstrated that Homo economicus, or preferences for exclusive maximization of individual material payoffs, is evolutionarily unstable . This idea also emerged earlier in other social sciences .
Within this issue, Alger and Lehmann focus on semi-Kantian morality preferences. The main novelty that Alger and Lehmann allow for is the ability of individuals to exhibit plastic behavior by adjusting their preference function depending on whom they are interacting with. Specifically, the authors consider three cases: incomplete information over types distribution, complete information and incomplete behavioral plasticity, and complete information and complete plasticity. They find that in the absence of information, the Kantian coefficient is equal to the coefficient of neutral relatedness between interacting individuals. However, complete information results in richer strategic choices that depend on demographic and interaction assumptions. Plasticity in this case allows for multiple uninvadable types, including the type whereby an individual exhibits flexible morality depending on whom they are interacting with.
Apart from moral considerations, preferences may also play a role in coordination problems. Within our special issue, Staab analyzes a two-action anti-coordination game where individuals benefit from choosing opposite actions. When decisions are made simultaneously, this requires interacting players to predict the behavior of their opponent in order to select a winning strategy and avoid costly miscoordination. Staab derives a preference over consumption lotteries when information about individual consumption is available. When individuals use relative consumption as a communication device, this can give rise to status preferences where higher-status individuals achieve better outcomes.