MultiModal Robotics

MultiSensory Robotics with MultiModal Abilities

June 21st,

OHE 100C, USC

Topics Covered

This workshop focuses on advancing robotics through the integration of multisensory and multimodal capabilities, extending beyond traditional visual perception and proprioception to include tactile sensing, auditory signals, language reasoning abilities, and other sensory modalities. The goal is to explore how robots can leverage diverse sensory inputs to enable robust operation in unstructured, real-world environments. By integrating multimodal data streams—such as vision, touch, sound, and proprioception—robots can develop a more comprehensive understanding of their surroundings, adapt to dynamic conditions, and interact more naturally with humans and their environment. This capability is critical for enabling "in-the-wild" operation, where robots must navigate complex, unpredictable scenarios with limited prior knowledge. Topics will include sensor fusion, multimodal learning, and the development of algorithms that enable robots to process and integrate heterogeneous data streams for improved autonomy and adaptability. Our workshop aims to highlight and discuss recent trends and advancements in sensing and reasoning in embodied intelligence. We target a core audience with interests in multimodal representation learning, causality reasoning, foundation models, and robotics. Presenters and panelists will be drawn from the following sub-communities:

General Robotics and Automation: Experts in robotic systems, sensor integration, and autonomous control
Sensory Fabrication and Manufacturing: Researchers in sensor design and fabrication towards robots needs
Multimodal Perception: Experts in integrating multimodal sensory modalities, such as tactile or auditory inputs, for enhanced perception
Multimodal Reasoning for Embodied Intelligence: Researchers working on agents that combine physical embodiment with advanced multimodal reasoning and language understanding.