Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand

Zilin Si*, Kevin Zhang*, Zeynep Temel, Oliver Kroemer

The Robotics Institute, Carnegie Mellon University

Overview

Tilde is an imitation learning-based in-hand manipulation system on a dexterous DeltaHand. It leverages 1) a low-cost, configurable, simple-to-control, soft dexterous robotic hand, DeltaHand, 2) a user-friendly, precise, real-time teleoperation interface, TeleHand, and 3) an efficient and generalizable imitation learning approach with diffusion policies. Our proposed TeleHand has a kinematic twin design to the DeltaHand that enables precise one-to-one joint control of the DeltaHand during teleoperation. This facilitates efficient high-quality data collection of human demonstrations in the real world. To evaluate the effectiveness of our system, we demonstrate the fully autonomous closed-loop deployment of diffusion policies learned from demonstrations across seven dexterous manipulation tasks with an average 90% success rate.

RSS-Tilde-video.mp4

Design

All design files can be found here.

DeltaHand

Please refer to the design from DELTAHANDS .

TeleHand

Overview of all the components used for making a TeleHand.

Tutorial for assembling a TeleHand.

assem-all.mp4

BibTex

@article{si2024tilde,

title={Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand},

author={Si, Zilin and Zhang, Kevin Lee and Temel, Zeynep and Kroemer, Oliver},

journal={arXiv preprint arXiv:2405.18804},

year={2024}

}

Contact: Zilin Si, zsi at andrew dot cmu dot edu

Page updated

Google Sites

Report abuse