Learning Quadrupedal Locomotion with Impaired Joints
Using Random Joint Masking
Mincheol Kim, Ukcheol Shin, Jung-Yup Kim
IEEE International Conference on Robotics and Automation (ICRA) 2024
HUMANOID ROBOT RESEARCH LAB(HRR LAB)
Seoul National University of Science and Technology
Abstract
Quadrupedal robots have played a crucial role in various environments, from structured environments to complex harsh terrains, thanks to their agile locomotion ability. However, these robots can easily lose their locomotion functionality if damaged by external accidents or internal malfunctions. In this paper, we propose a novel deep reinforcement learning framework to enable a quadrupedal robot to walk with impaired joints. The proposed framework consists of three components: 1) a random joint masking strategy for simulating impaired joint scenarios, 2) a joint state estimator to predict an implicit status of current joint condition based on past observation history, and 3) progressive curriculum learning to allow a single network to conduct both normal gait and various joint-impaired gaits. We verify that our framework enables the Unitree's Go1 robot to walk under various impaired joint conditions in real-world indoor and outdoor environments.
ICRA 2024 supplementary video
Method Overview
Our proposed method aims to learn quadrupedal locomotion with impaired locomotion. For this purpose, we first enforce impaired joint scenarios to the robot by randomly masking joint action at (Sec. III-B). Also, to make the robot estimate the current joint status (i.e., whether all joints are available or partially impaired) through observation history, we perform imitation learning between teacher and student latent vectors ˆzt and zt (Sec. III-C). Additionally, we propose a progressive curriculum learning strategy to enable comprehensive locomotion ability for a normal joint and various impaired joint conditions (Sec. III-D). Based on the proposed components, the robot is able to walk in not only a normal joint condition but also various impaired joint conditions.