3D Deformable Object Manipulation using Deep Neural Networks

Deformable object manipulation is a challenging problem in robotic manipulation. In this paper, we present a deep neural network based controller to servo-control the position and shape of deformable objects with unknown deformation properties. The servo-control is achieved by online learning a model mapping between the robotic end-effector's movement and the object's deformation measurement. In order to deal with the highly nonlinear properties of deformable objects, we use a multi-layer neural network to model the mapping function. In addition, we introduce a novel feature to describe deformable objects' deformation used in visual-servoing. This feature is directly extracted from the 3D point cloud rather from the 2D image as in previous work. In addition, we perform simultaneous tracking and reconstruction for the deformable object to resolve the partial observation problem during the deformable object manipulation.

We validate the performance of our algorithm and controller on a set of deformable object manipulation tasks and demonstrate that our method can achieve effective and accurate servo-control for general deformable objects with a wide variety of goal settings.