DreamWaQ
Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning
I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung
Urban Robotics Laboratory
School of Electrical Engineering, KAIST
Highlights
Team DreamSTEP of KAIST won the 1st place ICRA 2023 Autonomous Quadrupedal Robot Challenge! DreamWaQ was used as the locomotion controller for Team DreamSTEP's robot and able to traverse through challenging terrains in the competition!
Media Coverage
Abstract
Quadrupedal robots resemble the physical ability of legged animals to walk through unstructured terrains. However, designing a controller for quadrupedal robots poses a significant challenge due to their functional complexity and limited adaptability to various terrains. Recently, deep reinforcement learning, inspired by how legged animals learn to walk through their experiences, has been utilized to synthesize natural quadrupedal locomotion. However, state-of-the-art methods strongly depend on a complex and reliable sensing framework. Furthermore, prior works that rely only on proprioception have shown a limited demonstration for overcoming challenging terrains, especially long distances. This work proposes a novel quadrupedal locomotion learning framework that allows quadrupedal robots to walk through challenging terrains, even with limited sensing modalities. The proposed framework was validated in real-world outdoor environments with varying conditions within a single run for a long distance.
Overview
DreamWaQ trains a policy network to implicitly imagine the surrounding terrains only from proprioceptive measurements. Additionally, a context-aided estimator (CENet) is leveraged to improve robustness and adaptability of the learned policy
Experiments
Deformable slopes
On deformable slopes, DreamWaQ's policy can rapidly adapt to uncertainties such as foot slipping.
Thick vegetation
When faced with thick vegetation, DreamWaQ's policy can adjust its joint power to overcome trapping due to the vegetation.
Pedestrian curb
DreamWaQ's policy can easily adapt its gait pattern to overcome a 15cm-height pedestrian curb
Stairs
On medium-rise stairs, DreamWaQ's policy enables a relatively small A1 robot to go down and up the stairs without any mode change
Scalability
DreamWaQ is also easily scalable to different legged robots, ranging from small to big quadrupeds and a humanoid robot.
For all quadrupeds, there are no tuning in the reward functions and their corresponding scales.
Outdoor Long-walk Course
The experiment was conducted after rainfall, which made the stairs more slippery. Moreover, some natural terrains also turned into mud, making the robot's feet stepped deeper on it.
The hiking course has an elevation gain of up to 22m from the robot's starting point and a maximum slope of up to 36 degree.