MILE: A Mechanically Isomorphic Exoskeleton Data Collection System with Fingertip Visuotactile Sensing for Dexterous Manipulation 

Abstract

Imitation learning provides a promising approach to dexterous hand manipulation, but its effectiveness is limited by the lack of large-scale, high-fidelity data. Existing data-collection pipelines suffer from inaccurate motion retargeting, low data-collection efficiency, and missing high-resolution fingertip tactile sensing. We address this gap with MILE, a mechanically isomorphic teleoperation and data-collection system co-designed from human hand to exoskeleton to robotic hand. The exoskeleton is anthropometrically derived from the human hand, and the robotic hand preserves one-to-one joint-position isomorphism, eliminating nonlinear retargeting and enabling precise, natural control. The exoskeleton achieves a multi-joint mean absolute angular error below one degree, while the robotic hand integrates compact fingertip visuotactile modules that provide high-resolution tactile observations. Built on this retargeting-free interface, we teleoperate complex, contact-rich in-hand manipulation and efficiently collect a multimodal dataset comprising high-resolution fingertip visuotactile signals, RGB–D images, and joint positions. The teleoperation pipeline achieves a mean success rate improvement of 64%. Incorporating fingertip tactile observations further increases the success rate by an average of 25% over the vision-only baseline, validating the fidelity and utility of the dataset.

Hardware Design

co-designed from human hand to exoskeleton to robotic hand

System Performance Evaluation

Testeleoperation Demonstration

Inference & Ablation Experiment

Acknowledgement

We would like to thank Yunfan Zhang, Qianyou Zhao, Longyan Wu, Yueshi Dong, Yongyao Li, Xu Song and Zheng Wang for their invaluable advice on hardware design and learning policies. We also appreciate Jiapeng he, Nianzu Lv, Yutong Pei, Jinnuo Zhang, Zhenle Liu, and Yang Li for their assistance with data collection and user study. Lastly, we extend our appreciation to Boyang Peng  and Junjie Xia for their help in creating graphic renderings of the hardware.