AutocurriculaLab

A Multi-agent Reinforcement Learning Study of Libertarian and Utilitarian Governing Systems

It is generally believed that humans’ behaviours co-evolve with their governing systems. Governing systems or institutions could be mapped across the procedural-consequentialist axis from Full-Libertarian to Semi-Libertarian/Utilitarian, and from the latter to Full-Utilitarian systems, or across its discriminative nature from Inclusive to Arbitrary, and from the latter to Extractive institutions. In this study, by extending the AI-Economist - a recently developed two-level multi-agent reinforcement learning environment, by voting mechanism, first, it is shown that across the procedural-consequentialist axis, the Full-Libertarian governing system generates more inequity averse individuals. Additionally, it is shown that while under the Full-Libertarian governing system the Equality is lower, the Productivity and Maximin are higher. Finally, it is shown that resource sustainability is higher under the Full-Libertarian governing system. Afterward, by slightly modifying the voting mechanism, the Semi-Libertarian/Utilitarian governing system is divided to three governing institutions across its discriminative axis: Inclusive, Arbitrary, and Extractive. Then, it is shown that agents under the Arbitrary and Extractive institutions are less inequity averse compared to agents under an Inclusive institution. Furthermore, an Arbitrary institution is the least effective institution considering Productivity, Equality, and Maximin in the society. Moreover, while the resource sustainability is not significantly different across three governing institutions, by introducing a measure to calculate the fairness of an institution, it is shown that the Arbitrary and Extractive institutions are the most unfair systems. Overall, this paper adds to the growing literature of the application of multi-agent reinforcement learning in investigation of behavioral and economical phenomena.

Code: https://github.com/aslansd/modified-ai-economist 

Paper: https://drive.google.com/file/d/12KVWqOAMNX6rmr8xh0K8hlTKfPps3Bj-/view?usp=sharing 

A Multi-agent Reinforcement Learning Study of Emergence of Social Classes out of Arbitrary Governance: The Role of Environment

There are several theories in economics regarding the roots or causes of prosperity in a society. One of these theories or hypotheses - named geography hypothesis - mentions that the reason why some countries are prosperous and some others are poor is the geographical location of the countries in the world as makes their climate and environment favorable or unfavorable regarding natural resources. Another competing hypothesis states that man-made institutions particularly inclusive political institutions are the reasons why some countries are prosperous and some others are in poverty. On the other hand, there is a specific political theory developed for the long-term social development in Iran - named Arbitrary Rule and Aridisolatic Society which particularly emphasizes on the role of aridity to shape arbitrary political and economical institutions in Iran without any functional social classes in the society. In this paper, by extending the AI-Economist - a recently developed two-level multi-agent reinforcement learning environment, I show that when the central planner ruling the environment by arbitrary rules, the society evolves through different paths in different environments. In the environment having band-like vertical isolated patches of natural resources, all mobile agents are equally exploited by the central planner and the central planner is also not gaining any income, while in the society having more uniformly distributed natural resources, the productivity and Maximin are higher and the society generates a heterogeneous stratified social structure. All these findings provide a partial answer to the above debate and reconcile the role of geography and political institutions on the long-term development in a region.

Code: https://github.com/aslansd/modified-ai-economist 

Paper: https://drive.google.com/file/d/1T2ukmEeNB9NaML5fAQqTGgche6H6UOF1/view?usp=sharing 

A Multi-agent Reinforcement Learning Study of Evolution of Communication and Teaching under Libertarian and Utilitarian Governing Systems

Laboratory experiments have shown that communication plays an important role in solving social dilemmas. Here, by extending the AI-Economist, a mixed motive multi-agent reinforcement learning environment, I intend to find an answer to the following descriptive question: which governing system does facilitate the emer- gence and evolution of communication and teaching among agents? To answer this question, the AI-Economist is extended by a voting mechanism to simulate three different governing systems across individualistic-collectivistic axis, from Full-Libertarian to Full-Utilitarian governing systems. In the original framework of the AI-Economist, agents are able to build houses individually by collecting mate- rial resources from their environment. Here, the AI-Economist is further extended to include communication with possible misalignment –a variant of signaling game –by letting agents to build houses together if they are able to name mutually com- plement material resources by the same letter. Moreover, another extension is made to the AI-Economist to include teaching with possible misalignment –again a variant of signaling game –by letting half the agents as teachers who know how to use mutually complement material resources to build houses but are not capable of building actual houses, and the other half as students who do not have this information but are able to actually build those houses if teachers teach them. I found a strong evidence that collectivistic environment such as Full-Utilitarian system is more favourable for the emergence of communication and teaching, or more precisely, evolution of language alignment. Moreover, I found some evidence that evolution of language alignment through communication and teaching under collectivistic governing systems makes individuals more advantageously inequity averse. As a result, there is a positive correlation between evolution of language alignment and equality in the society.

Code: https://github.com/aslansd/modified-ai-economist-wc 

           https://github.com/aslansd/modified-ai-economist-wt 

Paper: https://drive.google.com/file/d/1Tz5spfNeCmmC1ZFrkxa83CpKrrKKKEaT/view?usp=sharing