Learning Robust Real-World Dexterous Grasping
Policies via Implicit Shape Augmentation
CoRL 2022
Zoey Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, Dieter Fox
University of Washington, Nvidia
Embedded Files
Abstract:
Dexterous robotic hands have the capability to interact with a wide variety of household objects to perform tasks like grasping. However, learning robust real world grasping policies for arbitrary objects has proven challenging due to the difficulty of generating high quality training data. In this work, we propose a learning system (ISAGrasp) for leveraging a small number of human demonstrations to bootstrap the generation of a much larger dataset containing successful grasps on a variety of novel objects. Our key insight is to use a correspondence-aware implicit generative model to deform object meshes and demonstrated human grasps in order to generate a diverse dataset of novel objects and successful grasps for supervised learning, while maintaining semantic realism. We use this dataset to train a robust grasping policy in simulation which can be deployed in the real world. We demonstrate grasping performance with a four-fingered Allegro hand in both simulation and the real world, and show this method can handle entirely new semantic classes and achieve a 79% success rate on grasping unseen objects in the real world.
Demo Video:
ISAGrasp_new.mp4Augmented Dataset via ISAGrasp (4000 objects) (slide to view more):
Real World Experiments (22 Unseen Objects with random poses)
Simulation Results
GoogleScan (200 unseen objects)
ShapeNet (200 unseen objects)
RescaledYCB (65 seen but rescaled objects)
Network:
Supplementary Video:
CoRL_ISAGrasp.mp4Implicit Shape Augmentation
We visualize some generated objects deformed from the original meshes.
Original Meshes
Deformation
Shape Deformation:
We visualize how shapes change along dim0, dim28, and dim 78 in a 128-dim latent vector using a power drill as an example.
dim78
dim 28
dim 0
Page updated
Google Sites
Report abuse