HIRO Hand
A Wearable Robotic Hand for Hand-over-Hand Imitation Learning
Dehao Wei and Huazhe Xu*
Abstract
Dexterous manipulation through imitation learning has gained significant attention in robotics research. The collection of high-quality expert data holds paramount importance when using imitation learning. The existing approaches for acquiring expert data commonly involve utilizing a data glove to capture hand motion information. However, this method suffers from limitations as the collected information cannot be directly mapped to the robotic hand due to discrepancies in their degrees of freedom or structures. Furthermore, it fails to accurately capture force feedback information between the hand and objects during the demonstration process. To overcome these challenges, this paper presents a novel solution in the form of a wearable dexterous hand, namely Hand-over-hand Imitation learning wearable RObotic Hand (HIRO Hand), which integrates expert data collection and enables the implementation of dexterous operations. This HIRO Hand empowers the operator to utilize their own tactile feedback to determine appropriate force, position, and actions, resulting in more accurate imitation of the expert’s actions. We develop both non-learning and visual behavior cloning based controllers allowing HIRO Hand successfully achieves grasping and in-hand manipulation ability.
Methods
HIRO hand comprises a hand glove, five mechanical fingers, fifteen motors, and a palm. The design of each finger is humanoid, with 4 humanoid joints and 4 humanoid tendon wires, where the coupling wire is used as a PIP-DIP coupling.
The wearable HIRO Hand allows us to collect dexterous manipulation data hand over hand easily with the potentiometer collecting each joint position and the camera collecting visual environment information.
Overview of the imitation learning experimental setup. (a) The experimental setup comprises a step motor, a linear guide with a valid travel distance of 450 mm, a base connector, the HIRO Hand , and a steel frame. (b) An overview of the training and testing process is presented, where hand-over-hand teaching videos and 15 potentiometer values are used as inputs and outputs of a convolutional neural network (CNN) consisting of five convolutional and two fully connected layers. The output value is binary, with 0 representing a command to stop the motor and 1 representing a command to move the motor. (c) The grasping capabilities of the HIRO Hand are demonstrated for different handle types, including "egg crate" foam, a cup, and a wire ball, and in-hand manipulation tasks, such as rotating the egg foam and grasping and unscrewing a faucet, using the behavior cloning algorithm.
Results
grasp egg foam with imitation learning
Rotate egg foam with imitation learning
Grasp and unscrew the faucet
Grasp the cup with Imitation learning
Grasp the cup with imitation learning
Teaching HIRO hand typing the letter "GREAT"
grasp ability test
Conclusion
MAIN CONTRIBUTION:
1. A wearable robotic hand with high precision and high pulling force was developed at low cost.
2. visual imitation learning-based controllers were developed for the wearable hand. It was trained to perform grasping and operating tasks through hand demonstrations.