AutocurriculaLab

A Multi-agent Reinforcement Learning Study of Libertarian and Utilitarian Governing Systems

It is generally believed that humans’ behaviours co-evolve with their governing systems. Governing systems or institutions could be mapped across the procedural-consequentialist axis from Full-Libertarian to Semi-Libertarian/Utilitarian, and from the latter to Full-Utilitarian systems, or across its discriminative nature from Inclusive to Arbitrary, and from the latter to Extractive institutions. In this study, by extending the AI-Economist - a recently developed two-level multi-agent reinforcement learning environment, by voting mechanism, first, it is shown that across the procedural-consequentialist axis, the Full-Libertarian governing system generates more inequity averse individuals. Additionally, it is shown that while under the Full-Libertarian governing system the Equality is lower, the Productivity and Maximin are higher. Finally, it is shown that resource sustainability is higher under the Full-Libertarian governing system. Afterward, by slightly modifying the voting mechanism, the Semi-Libertarian/Utilitarian governing system is divided to three governing institutions across its discriminative axis: Inclusive, Arbitrary, and Extractive. Then, it is shown that agents under the Arbitrary and Extractive institutions are less inequity averse compared to agents under an Inclusive institution. Furthermore, an Arbitrary institution is the least effective institution considering Productivity, Equality, and Maximin in the society. Moreover, while the resource sustainability is not significantly different across three governing institutions, by introducing a measure to calculate the fairness of an institution, it is shown that the Arbitrary and Extractive institutions are the most unfair systems. Overall, this paper adds to the growing literature of the application of multi-agent reinforcement learning in investigation of behavioral and economical phenomena.

Code: https://github.com/aslansd/modified-ai-economist 

Paper: https://drive.google.com/file/d/12KVWqOAMNX6rmr8xh0K8hlTKfPps3Bj-/view?usp=sharing 

A Multi-agent Reinforcement Learning Study of Emergence of Social Classes out of Arbitrary Governance: The Role of Environment

There are several theories in economics regarding the roots or causes of prosperity in a society. One of these theories or hypotheses - named geography hypothesis - mentions that the reason why some countries are prosperous and some others are poor is the geographical location of the countries in the world as makes their climate and environment favorable or unfavorable regarding natural resources. Another competing hypothesis states that man-made institutions particularly inclusive political institutions are the reasons why some countries are prosperous and some others are in poverty. On the other hand, there is a specific political theory developed for the long-term social development in Iran - named Arbitrary Rule and Aridisolatic Society which particularly emphasizes on the role of aridity to shape arbitrary political and economical institutions in Iran without any functional social classes in the society. In this paper, by extending the AI-Economist - a recently developed two-level multi-agent reinforcement learning environment, I show that when the central planner ruling the environment by arbitrary rules, the society evolves through different paths in different environments. In the environment having band-like vertical isolated patches of natural resources, all mobile agents are equally exploited by the central planner and the central planner is also not gaining any income, while in the society having more uniformly distributed natural resources, the productivity and Maximin are higher and the society generates a heterogeneous stratified social structure. All these findings provide a partial answer to the above debate and reconcile the role of geography and political institutions on the long-term development in a region.

Code: https://github.com/aslansd/modified-ai-economist 

Paper: https://drive.google.com/file/d/1T2ukmEeNB9NaML5fAQqTGgche6H6UOF1/view?usp=sharing 

A Multi-agent Reinforcement Learning Study of Evolution of Communication and Teaching under Libertarian and Utilitarian Governing Systems

Laboratory experiments have shown that communication plays an important role in solving social dilemmas. Here, by extending the AI-Economist, a mixed motive multi-agent reinforcement learning environment, I intend to find an answer to the following descriptive question: which governing system does facilitate the emer- gence and evolution of communication and teaching among agents? To answer this question, the AI-Economist is extended by a voting mechanism to simulate three different governing systems across individualistic-collectivistic axis, from Full-Libertarian to Full-Utilitarian governing systems. In the original framework of the AI-Economist, agents are able to build houses individually by collecting mate- rial resources from their environment. Here, the AI-Economist is further extended to include communication with possible misalignment –a variant of signaling game –by letting agents to build houses together if they are able to name mutually com- plement material resources by the same letter. Moreover, another extension is made to the AI-Economist to include teaching with possible misalignment –again a variant of signaling game –by letting half the agents as teachers who know how to use mutually complement material resources to build houses but are not capable of building actual houses, and the other half as students who do not have this information but are able to actually build those houses if teachers teach them. I found a strong evidence that collectivistic environment such as Full-Utilitarian system is more favourable for the emergence of communication and teaching, or more precisely, evolution of language alignment. Moreover, I found some evidence that evolution of language alignment through communication and teaching under collectivistic governing systems makes individuals more advantageously inequity averse. As a result, there is a positive correlation between evolution of language alignment and equality in the society.

Code: https://github.com/aslansd/modified-ai-economist-wc 

           https://github.com/aslansd/modified-ai-economist-wt 

Paper: https://drive.google.com/file/d/1Tz5spfNeCmmC1ZFrkxa83CpKrrKKKEaT/view?usp=sharing 

Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model

It has been shown that social institutions impact human motivations to produce different behaviours. Governing system as one major social institution is able to incentivise people in a society to work more or less, specialise labor in one specific field, or diversify their types of earnings. Until recently, this type of investigation is normally performed via economists by building mathematical models or performing experiments in the field. However, with advancement in artificial intelligence (AI), now it is possible to perform in-silico simulations to test various hypotheses around this topic. Here, in a curiosity-driven project, I simulate two somewhat similar worlds using multi-agent reinforcement learning (MARL) framework of the AI-Economist and generative agent-based model (GABM) framework of the Concordia. The AI-Economist is a two-level MARL framework originally devised for simulating tax behaviours of agents in a society governed by a central planner. Here, I extend the AI-Economist so the agents beside being able to build houses using material resources of the environment, would be able to trade their built houses, or trade their house building skill. Moreover, I equip the agents and the central planner with a voting mechanism so they would be able to rank different material resources in the environment. As a result of these changes, I am able to generate two sets of governmental types. Along the individualistic-collectivists axis, I produce a set of three governing systems: Full-Libertarian, Semi-Libertarian/Utilitarian, and Full-Utilitarian. Additionally, I further divide the Semi-Libertarian/Utilitarian governing system along the discriminative axis to a set of three governing institutions: Inclusive, Arbitrary, and Extractive. Building on these, I am able to show that among three governing systems, under the Semi-Libertarian/Utilitarian one (closely resembling the current democratic governments in the world in which the agents vote and the government counts the votes of the agents and implements them accordingly), the ratios of building houses to trading houses and trading house building skill are higher than the rest. Similarly, among governing institutions of the Semi-Libertarian/Utilitarian governing system, under the Inclusive institution, the ratios of building houses to trading houses and trading house building skill are higher than the rest. Moreover, the GABM framework of Concordia is originally devised to facilitate construction and use of generative agent-based models to simulate interactions of agents in grounded social space. The agents of this framework perform actions using natural language of large language models (LLMs), and a special agent called game master (which its role is similar to the central planner in the AI-Economist) translates their actions into appropriate implementations. I extended this framework via a component considering the inventory, skill, build, vote, and tax of the agents simultaneously, to generate similar three governing systems as above: Full-Libertarian, Semi-Libertarian/Utilitarian, and Full-Utilitarian. Among these governing systems, when the game master cares about equality in the society, it seems that under the Full-Utilitarian one, the agents build more houses and trade more house building skill. In contrast, when the game master cares about productivity in the society, under the Full-Libertarian governing system, it seems that the agents simultaneously build more houses, trade more houses, and trade more house building skill. Overall, the focus of this paper is on exploratory modelling and comparison of the power of two advanced techniques of AI, MARL and GABM, to simulate a similar social phenomena with limitations. Thus its main findings need further evaluation to be strengthen.

Code: https://github.com/aslansd/modified-ai-economist-gabm 

           https://github.com/aslansd/modified-concordia-marl 

Paper: https://drive.google.com/file/d/18rxxQiXvXzZAZKzRz39sD77t2u6YnCcW/view?usp=drive_link