Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand

Overview

Tilde is an imitation learning-based in-hand manipulation system on a dexterous DeltaHand. It leverages 1) a low-cost, configurable, simple-to-control, soft dexterous robotic hand, DeltaHand, 2) a user-friendly, precise, real-time teleoperation interface, TeleHand, and 3) an efficient and generalizable imitation learning approach with diffusion policies. Our proposed TeleHand has a kinematic twin design to the DeltaHand that enables precise one-to-one joint control of the DeltaHand during teleoperation. This facilitates efficient high-quality data collection of human demonstrations in the real world. To evaluate the effectiveness of our system, we demonstrate the fully autonomous closed-loop deployment of diffusion policies learned from demonstrations across seven dexterous manipulation tasks with an average 90% success rate. 

Design

All design files can be found here.

DeltaHand

Please refer to the design from DELTAHANDS .

TeleHand

Overview of all the components used for making a TeleHand.

Tutorial for assembling a TeleHand.

BibTex

@article{si2024tilde,

  title={Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand},

  author={Si, Zilin and Zhang, Kevin Lee and Temel, Zeynep and Kroemer, Oliver},

  journal={arXiv preprint arXiv:2405.18804},

  year={2024}

}

Contact: Zilin Si, zsi at andrew dot cmu dot edu