Logan Cross, Stanford University

Abstract:

 Multi-agent reinforcement learning (MARL) methods struggle with the non-stationarity of multi-agent systems and fail to adaptively learn online when tested with novel agents. Here, we leverage large language models (LLMs) to create an autonomous agent that can handle these challenges. Our agent, the Hypothetical Minds model, scaffolds the decision-making process around an LLM-mediated Theory of Mind module. This module enables the agent to synthesize hypotheses about its opponent’s strategy and goals. It then iteratively evaluates and refines these hypotheses, drawing on its memory of prior events, and leveraging intrinsic rewards derived from the LLM’s own predictions. Hypothetical Minds significantly improves performance over RL baselines on the challenging Running With Scissors scenario in the Melting Pot MARL benchmark. In contrast to RL methods that are trained with a large number of samples, the Hypothetical Minds agent succeeds in a zero-shot fashion, learning to identify and exploit strategies purely from in-context learning.