DefGoalNet: Contextual Goal Learning from Demonstrations For Deformable Object Manipulation
ABSTRACT:
Shape servoing, a robotic task dedicated to controlling objects to desired goal shapes, is a promising approach to deformable object manipulation. An issue arises, however, with the reliance on the specification of a goal shape. This goal has been obtained either by a laborious domain knowledge engineering process or by manually manipulating the object into the desired shape and capturing the goal shape at that specific moment, both of which are impractical in various robotic applications. In this paper, we solve this problem by developing a novel neural network DefGoalNet, which learns deformable object goal shapes directly from a small number of human demonstrations. We demonstrate our method's effectiveness on various robotic tasks, both in simulation and on a physical robot. Notably, in the surgical retraction task, even when trained with as few as 10 demonstrations, our method achieves a median success percentage of nearly 90%. These results mark a substantial advancement in enabling shape servoing methods to bring deformable object manipulation closer to practical, real-world applications.
Our paper is now available on arxiv! https://arxiv.org/abs/2309.14463
Method Overview:
DefGoalNet - Neural Network Architecture:
Videos:
Ground-truth (red) vs. Predicted Goal (others) -- Across multiple dataset sizes
Kidney-Conditioned Retraction
Tissue Wrapping
Hand-Condition Retraction
Experiment Results:
Code:
Currently unavailable for public
Dataset:
Currently unavailable for public
Other materials: