Imitation learning offers a pathway for robots to perform repetitive tasks, allowing humans to focus on more engaging and meaningful activities. However, challenges arise from the need for extensive demonstrations and the disparity between training and real-world environments. This paper focuses on contact-rich tasks involving deformable objects, such as wiping with a sponge, requiring adaptive force control to handle variations in wiping surface heights and the sponge's physical properties. 

To address these challenges, we propose a novel framework that integrates real-time time-series force-torque (FT) feedback with pre-trained object representations, allowing robots to dynamically adjust to previously unseen changes in wiping surface heights and sponge's physical properties using only a few human demonstrations. The method enables closed-loop force control considering object properties without requiring explicit target forces or position information, making it particularly suitable for deformable object manipulation.

🔗 arXiv  🔗 RA-L Version 

We propose a framework that combines pre-training to represent the physical properties of manipulated objects with real-time feedback of time-series force-torque (FT) information, enabling the robot’s adaptation to environmental changes from a small number of human demonstrations.

The proposed method consists of two steps:

• Pre-training step (simulation)

We pre-train the sponge property encoder using unlabeled FT trajectories, which are collected in simulation through predefined exploratory actions designed to expose variations in sponge's stiffness and friction. Using a VAE-based self-supervised framework, the encoder learns a latent representation capturing sponge's physical properties, which is then sim-to-real transferred.

• Training step (real robot)

We train a motion trajectory decoder to generate wiping motions based on sponge properties using few-shot imitation learning, and an FT feedback loop to predict the end-effector’s next vertical position from time-series FT history and sponge properties for adaptive control. 

In real-world experiments, our method achieved 96% accuracy in applying reference forces, significantly outperforming the previous method that lacked an FT feedback loop, which only achieved 4% accuracy. To evaluate the adaptability of our approach, we conducted experiments under different conditions from the training setup, involving 40 scenarios using 10 sponges with varying physical properties and 4 types of wiping surface heights, demonstrating significant improvements in the robot's adaptability by analyzing force trajectories.

@ARTICLE{10752344,

  author={Tsuji, Chikaha and Coronado, Enrique and Osorio, Pablo and Venture, Gentiane},

  journal={IEEE Robotics and Automation Letters}, 

  title={Adaptive Contact-Rich Manipulation Through Few-Shot Imitation Learning With Force-Torque Feedback and Pre-Trained Object Representations}, 

  year={2025},

  volume={10},

  number={1},

  pages={240-247},

  keywords={Robots;Force;Decoding;Training;Surface impedance;Trajectory;Force control;Feedback loop;Imitation learning;Impedance;Deep learning in grasping and manipulation;imitation learning;force control;representation learning},

  doi={10.1109/LRA.2024.3497713}}