DexPilot: Vision Based Tele-operation of Dexterous Robotic Hand-Arm System
Paper link: http://arxiv.org/abs/1910.03135
Highlights of the results
A selected list of demonstrations is shown on the left. Two pilots trained themselves to perform these tasks. The amount of training required is minimal --- pilots did 3-4 consecutive trails to warm-up before actually teleoperating the robot for the task --- and that it is possible to train new pilots quickly.
The teleoperation is driven by line of sight and no tactile feedback is relayed to the pilot. Despite that the pilots were able to perform a variety of tasks. Tactile feedback will only improve the overall process and is an important direction for future work.
Task: extract paper currency from a closed wallet (pilot 1 and pilot 2 alternate)
On the left, the pilots are able to teleoperate the robot hand-arm system for a challenging multi-step long-horizon task of pulling out paper currency from the wallet lying closed on the table. The hand is able to hold on to objects as thin as a paper between the fingers showing the robustness of kinematic retargeting.
DexPilot also enables two different pilots to alternate without requiring any additional calibration or change in the set-up. This feature was leveraged for this long horizon task to minimize pilot fatigue and discomfort.
Below are more examples of teleoperation with the DexPilot system. As shown, the system is sufficiently dexterous to solve tasks that require prehensile and non-prehensile manipulation, precision and power grasping, in-hand manipulation, and finger gaiting. Ultimately, rich sensorimotor responses are produced that could be used in the future to learn autonomous policies for complex tasks.