RL

Image credit

Contributors: Aleksandra Faust, Olivier Pietquin, Shiyu Huang, Wei-Wei Tu

Initial draft copied from Aleksandra's document, please elaborate.

RL benchmarks tend to be clustered based on the general application area. Here are the major benchmarks per category.

• Discrete Actions / Planning:

  • Arcade Learning Environment [Bellemare, et al., 2013]

  • Procgen [Cobbe et al., 2020]

  • Minigrid [Chevalier et al, 2018 ]

  • Honor of Kings [Wei et al., 2022 ]

  • BabyAI [Chevalier et al, 2019]

  • Gymnasium (replacement of OpenAI Gym)

• Continuous Actions / Controls:

  • Deepmind control suite

  • DiffRL [Xu et al., 2021 ]

  • Real World RL suite (RW4RL), [Dulac-Arnold et al., 2019] control benchmark with perturbations to evaluate robustness

  • Braxlines [Gu et al, 2021]

• Multi-agent Reinforcement Learning (MARL): -- reference paper

  • Google Research Football [Kurach et al. (2020) ]

  • Neural MMO [Joseph et al., 2019 ]

  • PettingZoo [Terry et al., 2020 ]

  • Fever Basketball [Hangtian et al., 2020 ]

  • Hanabi [Bard et al. (2020)]

  • Hide-And-Seek Domain [Baker et al. (2019)]

  • Multi-agent Particle World [Lowe et al. (2017)]

  • rSoccer [Felipe et al., 2021 ]

  • StarCraft II Micromanagement (SMAC) [Samvelyan et al. 2019]

  • OpenSpiel [Marc et al., 2019 ]

• RL for Unmanned Aerial Vehicle (UAV):

  • Gym-Pybullet-Drones [Jacopo et al., 2021 ]

  • AirSim [Shi et al., 2017 ]

  • Fast-Planner [Boyu et al., 2019 ]

• RL for Dialogue System/Natural Language Processing:

  • ScienceWorld [Ruoyao et al. 2022 ]

  • RL4LMs [Rajkumar et al., 2022 ]

  • The NetHack Learning Environment (NLE) [Heinrich et al., 2020 ]

• RL for Robots:

  • Isaac Gym [Viktor et al. 2021 ]

  • ASE [Xue et al., 2022 ]

  • RCareWorld [Ruolin et al., 2022 ]

• RL for Musculoskeletal Control:

  • MyoSuite [Vittorio et al., 2022 ]

  • MASS [Seunghwan et al., 2019 ]

  • OstrichRL[Vittorio et al. 2021 ]

• Embodied AI:

  • Avalon [Joshua et al., 2022 ]

  • Habitat 2.0 [Andrew et al., 2021 ]

• Complex / Compositional Tasks:

  • Compositional RL [Gur et al., 2021]

  • StarCraft II Learning Environment [Vinyals, et al, 2017]

• Offline RL:

  • RL Unplugged, [Gulcehre et al, 2021]

  • Data for RL (D4RL) [Fu et al., 2021]

  • NeoRL [Rongjun et al., 2021 ]

• Real-world-based:

• Balloon Learning Environment, [Bellemare et al., 2020]

• Chip Design, [Mirhoseini et al. 2021],

• Air Learning for aerial robots [Krishnan et al., 2021]

• GridAlive for Power Grids control [Marot et al, 2021]