MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffer
Jeewon Jeon, Woojun Kim, Whiyoung Jung, and Youngchul Sung
Presented at International Conference on Machine Learning (ICML) 2022, Baltimore, MD, Jul. 2022
Embedded Files
Abstract
In this paper, we consider cooperative multi-agent reinforcement learning (MARL) with sparse reward. To tackle this problem, we propose a novel method named MASER: MARL with subgoals generated from experience replay buffer. Under the widely-used assumption of centralized training with decentralized execution and consistent Q-value decomposition for MARL, MASER automatically generates proper subgoals for multiple agents from the experience replay buffer by considering both individual Q-value and total Q-value. Then, MASER designs individual intrinsic reward for each agent based on actionable representation relevant to Q-learning so that the agents reach their subgoals while maximizing the joint action value. Numerical results show that MASER significantly outperforms StarCraft II micromanagement benchmark compared to other state-of-the-art MARL algorithms.
Poster
Page updated
Google Sites
Report abuse