January 26th 2021: HiDe has been accepted for publication in the IEEE Robotics and Automation Letters (RA-L): https://ieeexplore.ieee.org/document/9357915

December 11th 2020: HiDe will be presented as a contributed talk at the Neurips Deep Reinforcement Learning Workshop on December 11th: sites.google.com/view/deep-rl-workshop-neurips2020

We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across the hierarchy, which allows the integration of task-relevant knowledge per layer. We propose an RL-based planner to efficiently leverage the information in the planning layer of the hierarchy, while the control layer learns a goal-conditioned control policy. The hierarchy is trained jointly but allows for the composition of different policies such as transferring layers across multiple agents. We experimentally show that our method generalizes across unseen test environments and can scale to tasks well beyond 3x horizon length compared to both learning and non-learning based approaches. We evaluate on complex continuous control tasks with sparse rewards, including navigation and robot manipulation.

Code & pretrained models: here

Paper PDF: arxiv.org/abs/2002.05954

Email: sammy.christen@inf.ethz.ch