Jianlan Luo*, Charles Xu*, Xinyang Geng, Gilbert Feng, Kuan Fang, Liam Tan, Stefan Schaal, Sergey Levine
UC Berkeley Intrinsic Innovation LLC
We present a system that performs precise multi-stage cable routing tasks through hierarchical imitation learning; where a high-level policy intelligently selects from a library of low-level primitives. The key idea is to have primitives that can compensate for others' deficiencies so that the system's overall performance will not solely hinge on that of each individual primitive. Our system exhibits robust recovery behavior in the face of failures; furthermore, it can rapidly adapt to new scenarios with an interactive fine-tuning scheme.
We study the problem of learning to perform multi-stage robotic manipulation tasks, with applications to cable routing, where the robot must route a cable through a series of clips. This setting presents challenges representative of complex multi-stage robotic manipulation scenarios: handling deformable objects, closing the loop on visual perception, and handling extended behaviors consisting of multiple steps that must be executed successfully to complete the entire task. In such settings, learning individual primitives for each stage that succeed with a high enough rate to perform a complete temporally extended task is impractical: if each stage must be completed successfully and has a non-negligible probability of failure, the likelihood of successful completion of the entire task becomes negligible. Therefore, successful controllers for such multi-stage tasks must be able to recover from failure and compensate for imperfections in low-level controllers by smartly choosing which controllers to trigger at any given time, retrying, or taking corrective action as needed. To this end, we describe an imitation learning system that uses vision-based policies trained from demonstrations at both the lower (motor control) and the upper (sequencing) level, present a system for instantiating this method to learn the cable routing task, and perform evaluations showing great performance in generalizing to very challenging clip placement variations.
The high-level primitive selection policy takes the robot wrist and side camera observations, as well as the history of executed primitives as input, and outputs a categorical distribution to select the next primitive. The low-level single clip cable routing policy only uses the wrist camera observations and the robot state, and outputs a Gaussian distribution of robot actions. This decomposition of our system into high-level and low-level policies allows us to collect data and train policies with large flexibility, thus enabling the robot to master sophisticated cable routing tasks while reducing the overall complexity of our system.
@article{luo2023multistage,
author = {Jianlan Luo and Charles Xu and Xinyang Geng and Gilbert Feng and Kuan Fang and Liam Tan and Stefan Schaal and Sergey Levine},
title = {Multi-Stage Cable Routing through Hierarchical Imitation Learning},
journal = {arXiv pre-print},
year = {2023},
url = {https://arxiv.org/abs/2307.08927},
}
Contact: jianlanluo [at] berkeley [dot] edu and xuc [at] berkeley [dot] edu.