Unity ML-Agents is one of the major open-source frameworks for reinforcement learning research. The repository https://github.com/Unity-Technologies/ml-agents. It is characterized by ingenious engineering, including efficient parallelized training procedures, many example environments, and implementations of state-of-the-art algorithms including PPO, SAC (single agent setting), and COMA (multi-agent).

In our research, we have been using the newest environment, the Unity Dodgeball https://github.com/Unity-Technologies/ml-agents-dodgeball-env

We base on the Fictitious Co-Play algorithm from DeepMind's DJ Strouse et al. Collaborating with Humans without Human Data (NeurIPS 2021) for multi-player & multi-team settings and evaluate it within the Dodgeball environment.

The algorithm is aimed at training agents that can generalize zero-shot to co-play with out-of-distribution teammates and even human teammates.

Originally FCP algorithm was tested in a very simple over-cooked environment, based on human-ai collaboration during a cooking task taking place on a simple 2D grid-world.

We generalize FCP to multi-player team-based 3D game.

For the purpose of performing Human-AI experiments, we built a hand-crafted FPP computer game mod of the Dodgeball environment, utilizing the Unity 3D game engine capabilities. The game is currently restricted to a single operating system, but we are currently developing an on-line game natively running through an internet browser.