Learning Subgoal Representations with Slow Dynamics

Siyuan Li* Lulu Zheng* Jianghao Wang Chongjie Zhang

Tsinghua University

[Paper] [Code]

In this paper, we propose slow dynamics as a novel learning objective for the subgoal representation in goal-conditioned hierarchical reinforcement learning. Here are the videos of our learned policies on a suite of complex MuJoCo tasks. The directive is to make the robot reach the goal denoted by a green block. Our method is able to learn successful hierarchical policies on top of the learned slow features. In addition, our learned representation and low-level policies are transferable between different tasks.

point_maze.mp4
ant_maze.mp4

Point Maze

Ant Maze

ant_push.mp4
ant_fall.mp4

Ant Push

Ant Fall

fourroom.mp4

Transfer from Ant Maze to Ant FourRooms