EduGym
An environment and notebook suite for reinforcement learning education
EduGym is an interactive suite for reinforcement learning education, intended as supplementary material to a standard RL course.
The core learning experience of EduGym are the interactive (Colab) notebooks. Each notebook focuses on a specific aspect of reinforcement learning, such as exploration, credit assignment, state dimensionality and partial observability, etc. Try the notebooks out here!
Each notebook (and the concept it discusses) also comes with its own environment, which is specifically designed to illustrate the particular concept/challenge. EduGym is therefore also environment package.
(The paper is currently under review, but EduGym and its environments be released as a Github package.)
Setting
Although there is a variety of reinforcement learning (RL) teaching material, we in practice identify a few remaining gaps:
Textbooks provide the essential introduction to the field, often with nice conceptual examples. However:
Students often struggle to translate between equations and code.
Public (documented) codebases are great for research. However:
They implement complicated algorithms that conflate many separate algorithmic ideas.
They often contain much boilerplate code (e.g., for logging), 'hiding' the key RL essentials.
Environment packages (based on Gymnasium) provide a unified framework for testing. However:
Most environments conflate several challenges (stochasticity, partial observability, exploration, etc.), and are not tailored to teach a particular aspect/challenge of RL.
Most environments are high-dimensional, and therefore not suitable for fast interactive experimentation.
Solution
The above problem creates a gap for:
Educational reinforcement learning environments, which are:
designed to illustrate a particular aspect/challenge of reinforcement learning.
(ideally) have tunable parameters to vary the strenght of that particular challenge.
allow for a fast experimentation cycle.
An associated set of interactive notebooks, which:
illustrate each challenge and possible solution approaches, connecting equations and code in a single document.
allow for (fast) interactive experimentation by the student.
EduGym
EduGym exactly implements the above solution:
We selected a set of core topics in reinforcement learning (see the list under Notebooks),
designed an environment that illustrates the particular challenge, and
made an interactive notebook that explains the challenge and possible solution approaches (using its associated environment).
As such, EduGym may be used as a companion to classic textbooks, (online) courses and existing documented RL codebases.
Check-out the environments, and all interactive notebooks!
If you have any feedback, please let us know at the below contact information.
(not available in current anonymous version)
We hope you enjoy EduGym!