Learning More With Less: Sample-Efficient Model-Based RL for Loco-Manipulation
Benjamin Hoffman, Jin Cheng, Chenhao Li, Stelian Coros
Computational Robotics Lab | Learning & Adaptive Systems Group | ETH AI Center | ETH Zurich, Switzerland
Embedded Files
Abstract
By combining the agility of legged locomotion with the capabilities of manipulation, loco-manipulation platforms have the potential to perform complex tasks in real-world applications. To this end, state-of-the-art quadrupeds with manipulators, such as the Boston Dynamics Spot, have emerged to provide a capable and robust platform. However, the complexity of loco-manipulation control, as well as the black-box nature of commercial platforms, pose challenges for deriving accurate dynamics models and control policies.
To address these challenges, we develop a hand-crafted kinematic model of a quadruped-with-arm platform which – employing recent advances in Bayesian Neural Network (BNN)–based learning – we leverage as a physical prior to efficiently learn an accurate dynamics model from limited data. We then derive control policies for loco-manipulation via model-based reinforcement learning (RL). We demonstrate the effectiveness of our approach on hardware using the Boston Dynamics Spot, accurately performing dynamic end-effector trajectory tracking even in low data regimes.
Overview
Evaluation
We demonstrate the effectiveness of our approach on the Boston Dynamics Spot, a state-of-the-art quadruped robot equipped with an arm for manipulation. We leverage our learned dynamics model to develop control policies for loco-manipulation via model-based RL and then use the learned policy to dynamically track two shapes with the robot’s end-effector: an ellipse and a helix.
We compare our main approach, SIM-FSVGD, to the two baseline models SIM-MODEL and FSVGD:
SIM-MODEL: Hand-crafted kinematic model from first principles with learnable parameters.
FSVGD: BNN-model that uses function space particle optimization and is widely applied in bayesian deep learning [2].
SIM-FSVGD: BNN-model based on FSVGD that uses our kinematic model as a physical prior to achieve better sample efficiency [1].
[1] J. Rothfuss, et al., “Bridging the Sim-to-Real Gap with Bayesian Inference,” 2024
[2] Z. Wang, et al., “Function Space Particle Optimization for Bayesian Neural Networks,” 2019
Ellipse Tracking
Helix Tracking
Page updated
Google Sites
Report abuse