DreamWaQ


Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning

I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung

Urban Robotics Laboratory

School of Electrical Engineering, KAIST

Highlights

Team DreamSTEP of KAIST won the 1st place ICRA 2023 Autonomous Quadrupedal Robot Challenge! DreamWaQ was used as the locomotion controller for Team DreamSTEP's robot and able to traverse through challenging terrains in the competition!

Abstract

Quadrupedal robots resemble the physical ability of legged animals to walk through unstructured terrains. However, designing a controller for quadrupedal robots poses a significant challenge due to their functional complexity and limited adaptability to various terrains. Recently, deep reinforcement learning, inspired by how legged animals learn to walk through their experiences, has been utilized to synthesize natural quadrupedal locomotion. However, state-of-the-art methods strongly depend on a complex and reliable sensing framework. Furthermore, prior works that rely only on proprioception have shown a limited demonstration for overcoming challenging terrains, especially long distances. This work proposes a novel quadrupedal locomotion learning framework that allows quadrupedal robots to walk through challenging terrains, even with limited sensing modalities. The proposed framework was validated in real-world outdoor environments with varying conditions within a single run for a long distance.

Overview

DreamWaQ trains a policy network to implicitly imagine the surrounding terrains only from proprioceptive measurements. Additionally, a context-aided estimator (CENet) is leveraged to improve robustness and adaptability of the learned policy

Experiments

Deformable slopes

On deformable slopes, DreamWaQ's policy can rapidly adapt to uncertainties such as foot slipping.

Thick vegetation

When faced with thick vegetation, DreamWaQ's policy can adjust its joint power to overcome trapping due to the vegetation.

Pedestrian curb

DreamWaQ's policy can easily adapt its gait pattern to overcome a 15cm-height pedestrian curb

Stairs

On medium-rise stairs, DreamWaQ's policy enables a relatively small A1 robot to go down and up the stairs without any mode change

Scalability

DreamWaQ is also easily scalable to different legged robots, ranging from small to big quadrupeds and a humanoid robot.

For all quadrupeds, there are no tuning in the reward functions and their corresponding scales.

Outdoor Long-walk Course

The experiment was conducted after rainfall, which made the stairs more slippery. Moreover, some natural terrains also turned into mud, making the robot's feet stepped deeper on it.

The hiking course has an elevation gain of up to 22m from the robot's starting point and a maximum slope of up to 36 degree.