Learning a Contact-Adaptive Controller for

Robust, Efficient Legged Locomotion

Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Animashree Anandkumar, Yuke Zhu, Buck Babich, Animesh Garg


Accepted to Conference on Robot Learning (CoRL), 2020

Abstract

We present a hierarchical framework that combines model-based control and reinforcement learning (RL) to synthesize robust controllers for a quadruped (the Unitree Laikago). The system consists of a high-level controller that learns to choose from a set of primitives in response to changes in the environment and a low-level controller that utilizes an established control method to robustly execute the primitives. Our framework learns a controller that can adapt to challenging environmental changes on the fly, including novel scenarios not seen during training. The learned controller is up to 85~percent more energy efficient and is more robust compared to baseline methods. We also deploy the controller on a physical robot without any randomization or adaptation scheme.

Contributions

  • We introduce a hierarchical control structure that combines model-based control design and model-free reinforcement learning for legged locomotion.

  • We demonstrate that our framework allows sample-efficient learning, zero-shot adaptation to novel scenarios, and direct sim-to-real transfer without randomization or adaptation schemes.

  • Our framework learns adaptive contact sequences that are not present in either model-based or learning-based methods in real-time control. This is evidenced in the natural-looking behaviors that minimize unnecessary movement and energy usage in our split-belt treadmill scenarios.

Overview of the system

Left: Primitives are distinguished by the contact configuration. The stance legs in each primitive are colored orange.

Center: Hierarchical structure of the controller. The high-level controller chooses from a set of primitives based on the robot state, and the low-level controller computes the motor torques based on the robot state and the primitive chosen.

Right: The low-level controller uses stance foot forces to control the base pose and moves the swing feet to their target positions.