Multi-fingered dexterous hands are versatile and capable of acquiring a diverse set of skills such as grasping, in-hand manipulation, and tool use. To fully utilize their versatility in real-world scenarios, we require algorithms and policies that can control them using on-board sensing capabilities, without relying on external tracking or motion capture systems. Cameras and tactile sensors are the most widely used on-board sensors that do not require instrumentation of the world. In this work, we demonstrate an imitation learning based approach to train deep visuomotor policies for a variety of manipulation tasks. These policies directly control the hand using high dimensional visual observations of the world and propreoceptive observations from the robot, and can be trained efficiently with a few hundred expert demonstration trajectories. We also find that using touch sensing information enables faster learning and better asymptotic performance for tasks with high degree of occlusions.

Approach

For robotic systems, although minor learning or adaptation might happen during deployment time, the majority of learning would happen in well instrumented laboratory settings and simulators. This means that although we may have access only to onboard sensing at deployment time, additional sensing capabilities or state information is available at training time. In this work, we formulate an approach that leverages this additional information to accelerate learning of deep visuomotor policies for dexterous manipulation tasks. Our approach consists of two stages:

• Using the full-state information (from instrumentation) to train a "compact" policy.
• Imitation learning to mimic the above compact expert policy, but using high-dimensional raw observations.

Overall, this two step procedure is vastly more efficient than learning visuomotor policies end-to-end. Furthermore, this approach has the same asymptotic performance as the expert policy and does not degrade in performance like a method that explicitly decouples state estimation and control.

Bibliography

@INPROCEEDINGS{Jain-ICRA-19,

    AUTHOR    = {Divye Jain AND Andrew Li AND Shivam Singhal AND 

                 Aravind Rajeswaran AND Vikash Kumar AND Emanuel Todorov},

    TITLE     = "{Learning Deep Visuomotor Policies for Dexterous Hand Manipulation}",

    BOOKTITLE = {International Conference on Robotics and Automation (ICRA)},

    YEAR      = {2019},

}