Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal Locomotion
Guillermo A. Castillo1, Bowen Weng1, Wei Zhang2, Ayonga Hereid1
1The Ohio State University, USA
2Southern University of Science and Technology, China
Guillermo A. Castillo1, Bowen Weng1, Wei Zhang2, Ayonga Hereid1
1The Ohio State University, USA
2Southern University of Science and Technology, China
This paper presents a novel model-free reinforcement learning (RL) framework to design cascade feedback control policies for 3D bipedal locomotion. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint or task space trajectories. Different from these studies, we propose a novel policy structure that properly incorporates physical insights from the nature of the walking dynamics and the well-established Hybrid Zero Dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, sample efficiency, and less dependence on prior knowledge. The proposed solution learns walking gaits from scratch and produces stable limit walking cycles that can track various walking speed in different directions. The learned policies also perform robustly against various adversarial forces applied to the torso and walking blindly on a series of challenging terrains. These results demonstrates the proposed cascade feedback control policy is suitable for navigation of bipedal robots in indoors an outdoors environments.
Full paper [IEEE Access]