Dynamic Manipulation Based on Artificial Vision and Touch

Human manipulation relies on a combination of dexterous hands, tactile sensing, and visual feedback to maintain controlled interactions with objects, even without a firm grasp, as in the case of a waiter carrying a tray. Recent robotic advances aim to replicate these skills through vision, tactile, and force sensors for both in-hand and whole-body manipulation, including multi-arm coordination. Key control strategies include object tracking, force control, internal force estimation, and non-prehensile manipulation.

Robots must modulate grasping forces to handle both rigid and deformable objects without damage. While parallel grippers dominate industrial applications due to their simplicity, recent research demonstrates that even these devices can perform complex tasks using extrinsic dexterity, for example, exploiting gravity to pivot or slide objects within the gripper.

In cluttered environments, intrinsic dexterity becomes critical. Direct grasping may be unfeasible, making non-prehensile actions like pushing or sliding more effective to reposition objects. Although more complex, such actions are part of our daily behavior and offer a broader range of manipulation strategies for robots.

Visual feedback supports object tracking and grasp planning in both prehensile and non-prehensile scenarios. However, visual sensing can be hindered by occlusions or lighting. Tactile sensing, while requiring contact, provides robust feedback. Combining both modalities can overcome their individual limitations, offering more adaptive and resilient control.

• Lorenzo Natale, Italian Institute of Technology. Title of the talk. Title of the talk: "Domain adaptation for learning tactile object features with vision-based sensors"

• Alessandro Marino, Department of Electrical and Information Engineering "Maurizio Scarano", Università degli Studi di Cassino e del Lazio Meridionale. Title of the talk: "Coordinated multi-arm control for manipulation in complex settings"

• Alessio Caporali, Dipartimento di Ingegneria dell'Energia Elettrica e dell'Informazione "Guglielmo Marconi", Alma Mater Studiorum Università di Bologna. Title of the talk: "Model-based Tracking of Deformable Linear Objects in Challenging Environments"

• Alessandro Saccon, Mechanical Engineering Department, Eindhoven University of Technology (TU/e). Title of the talk: "The Quest for Robust Contact-Rich and Impact-Aware Manipulation: On Control Theory, Physics Simulation, Robot Learning, and Suitable Hardware"

• Maria Pozzi, Dipartimento di Ingegneria dell'Informazione e Scienze Matematiche, Università di Siena. Title of the talk: "Soft manipulation: the role of mechanical intelligence in robotic grasping"

Acknowledgements