In-Hand Manipulation of Articulated Tools with Dexterous Robot Hands with Sim-to-Real Transfer

Left—open–close mechanics of scissors and pliers. Right—real-world and simulation grasp poses, showing alignment of digital clones with physical interactions.

Pipeline for articulated in-hand manipulation. (a) Oracle training with privileged observations in simulation, followed by distillation into a proprioceptive base policy and data collection with additional tactile and motor torque observations. (b) CATFA fuses these modalities with policy intent via cross-attention for online adaptation on hardware.