Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal Locomotion

Guillermo A. Castillo¹, Bowen Weng¹, Wei Zhang², Ayonga Hereid¹

¹The Ohio State University, USA

²Southern University of Science and Technology, China

Abstract

This paper presents a novel model-free reinforcement learning (RL) framework to design cascade feedback control policies for 3D bipedal locomotion. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint or task space trajectories. Different from these studies, we propose a novel policy structure that properly incorporates physical insights from the nature of the walking dynamics and the well-established Hybrid Zero Dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, sample efficiency, and less dependence on prior knowledge. The proposed solution learns walking gaits from scratch and produces stable limit walking cycles that can track various walking speed in different directions. The learned policies also perform robustly against various adversarial forces applied to the torso and walking blindly on a series of challenging terrains. These results demonstrates the proposed cascade feedback control policy is suitable for navigation of bipedal robots in indoors an outdoors environments.

Simulation and hardware results

Full paper [IEEE Access]

Page updated

Google Sites

Report abuse