Computing Ex Ante Equilibrium in Heterogeneous Zero-Sum Team Games
We run the approximate TMECor strategies against state-of-the-art Multi-Agent Reinforcement Learning (MARL) strategies and compare their relative winrate below.
Red Team: H-PSRO
Blue Team: MAT
Win Rate: 34.0
Red Team: Team PSRO
Blue Team: MAT
Win Rate: 7.0
Red Team: H-PSRO
Blue Team: HAPPO
Win Rate: 56.0
Red Team: Team PSRO
Blue Team: HAPPO
Win Rate: 7.0
Red Team: H-PSRO
Blue Team: MAPPO
Win Rate: 98.0
Red Team: Team PSRO
Blue Team: MAPPO
Win Rate: 100.0
Red Team: H-PSRO
Blue Team: MAT
Win Rate: 89.0
Red Team: Team PSRO
Blue Team: MAT
Win Rate: 18.0
Red Team: H-PSRO
Blue Team: HAPPO
Win Rate: 72.0
Red Team: Team PSRO
Blue Team: HAPPO
Win Rate: 1.0
Red Team: H-PSRO
Blue Team: MAPPO
Win Rate: 99.0
Red Team: Team PSRO
Blue Team: MAPPO
Win Rate: 100.0
Red Team: H-PSRO
Blue Team: MAT
Win Rate: 59.0
Red Team: Team PSRO
Blue Team: MAT
Win Rate: 20.0
Red Team: H-PSRO
Blue Team: HAPPO
Win Rate: 85.0
Red Team: Team PSRO
Blue Team: HAPPO
Win Rate: 10.0
Red Team: H-PSRO
Blue Team: MAPPO
Win Rate: 100.0
Red Team: Team PSRO
Blue Team: MAPPO
Win Rate: 95.0
[Competitive StarCraft] Leroy, Pascal, Jonathan Pisane, and Damien Ernst. "Value-based CTDE methods in symmetric two-team markov game: From cooperation to team competition." arXiv preprint arXiv:2211.11886 (2022).
[Team PSRO] McAleer, Stephen, et al. "Team-PSRO for learning approximate TMECor in large team games via cooperative reinforcement learning." Advances in Neural Information Processing Systems 36 (2023): 45402-45418.
[MAT] Wen, Muning, et al. "Multi-agent reinforcement learning is a sequence modeling problem." Advances in Neural Information Processing Systems 35 (2022): 16509-16521.
[HAPPO] Kuba, Jakub Grudzien, et al. "Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning." International Conference on Learning Representations.
[MAPPO] Yu, Chao, et al. "The surprising effectiveness of ppo in cooperative multi-agent games." Advances in Neural Information Processing Systems 35 (2022): 24611-24624.