Abstract

Despite recent successes of reinforcement learning (RL), it remains a common problem that agents fail to transfer their learned skills to related environments. To facilitate research addressing this challenge, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment. This environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strengths of CausalWorld is that it provides a combinatorial family of such tasks with a common causal structure and underlying factors (including e.g. robot and object masses, colors, sizes). The user (or even the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. Hence, one can easily define training and evaluation distributions of a desired difficulty level, targeting a desired form of generalization (e.g. only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to extremely challenging, all of which require long-horizon planning and precise low-level motor control at the same time. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.