Sun Zhaole^1, Xiao Gao^2, Xiaofeng Mao^1, Jihong Zhu^3, Aude Billard^2, and Robert B. Fisher^1
1. University of Edinburgh, 2. EPFL, 3. University of York
Paper link (Arxiv): https://arxiv.org/pdf/2502.00396
Video link (Youtube): https://www.youtube.com/watch?v=qOYxl7-azhE
Three long-horizon in-hand cable manipulation performed by our hand.
In-hand cable pulling with multiple primitives by our hand.
The taxonomy tree of Dexterous Cable Manipulation (Cable Dexonomy)
Humans use their hands to dexterously manipulate cables to perform various tasks, like grasping cables, moving cables in hand without dropping them, bending the cable into a U shape for hooking and so on. Existing research that addressed cable manipulation relied on two-fingered grippers, which make it difficult to perform similar cable manipulation tasks that humans perform. This is due to the limited dexterity of a two-fingered gripper, which can only grasp and release a cable without additional manipulability. Thus, we need a multi-fingered hand, which is much more dexterous than a two-fingered gripper. However, unlike dexterous manipulation of rigid objects, the development of dexterous cable manipulation skills in robotics remains underexplored due to the unique challenges posed by a cable's deformability and inherent uncertainty. In addition, using a dexterous hand introduces specific difficulties in tasks, such as cable grasping, pulling, and in-hand bending, for which no dedicated task definitions, benchmarks, or evaluation metrics exist. Furthermore, we observed that most existing dexterous hands are designed with structures identical to humans', typically featuring only one thumb, which often limits their effectiveness during dexterous cable manipulation. Lastly, existing non-task-specific methods did not have enough generalization ability to solve these cable manipulation tasks or are unsuitable due to the designed hardware.
We address these three challenges in real-world dexterous cable manipulation in the following steps:
(1) We first defined and organized a set of dexterous cable manipulation tasks into a comprehensive taxonomy, covering most short-horizon action primitives and long-horizon tasks for one-handed cable manipulation. This taxonomy revealed that coordination between the thumb and the index finger is critical for cable manipulation, which decomposes long-horizon tasks into simpler primitives.
(2) We designed a novel five-fingered hand with 25 degrees of freedom (DoF), featuring two symmetric thumb-index configurations and a rotatable joint on each fingertip, which enables dexterous cable manipulation.
(3) We developed a demonstration collection pipeline for this non-anthropomorphic hand, which is difficult to operate by previous motion capture methods.
Given only one demonstration on one specific cable for each manipulation, among 8 primitives, our method achieved 88% success rate of demonstration replaying on three cables of the same material but various diameters and over 75% success rate on three cables of very different materials, stiffness, and diameters. Based on collected primitive demonstrations, we developed finite state machines (FSM) that enabled the robotic hand to execute complex long-horizon tasks without requiring prior demonstrations of the complete trajectories. Among four long-horizon and complicated manipulations, we achieved a 64% success rate of demonstration replaying on the cables of the same materials and various diameters. Our robotic hand achieved performance comparable to human baseline dexterity in both primitive actions and long-horizon tasks. Through human-guided transitions between primitives, the robotic hand demonstrates robust long-horizon manipulation capabilities even under diverse external disturbances.