A Generative Neural Network for Learning Coordinated Reach-Grasp Motions

[ Information ]

IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2769-2776, 2019.

[ Authors ]

Eunsuk Chong, Jinhyuk Park, Hyungmin Kim, and Frank C. Park

[ Abstract ]

A neural network for generating coordinated reach-grasp motions is proposed, based on a type of generative neural network called the conditional restricted Boltzmann machine (CRBM). Given demonstrations of humans reaching and grasping various target objects of different shapes and poses, a mixture-type CRBM model is first used to learn and cluster the reach-grasp motions into different movement types. A novel variant of CRBM, called CRBM-l, is then proposed, in which the CRBM network is augmented with a control variable that is adjustable for different target objects. A CRBM-l trained with the previously obtained movement-specific data is then used to generate real-time reach-grasp motions for new target objects, by appropriately adjusting the control variable. The generated reach-grasp motions are then fine-tuned taking into account the contact states between the object and the hand/fingers. The versatility and efficiency of our reach-grasp motion generation method is validated through systematic experiments involving a diverse set of target objects.