Generative Modeling meets HRI

The rapid evolution of generative modeling has led to tremendous advances in recent years, enabling breakthroughs across a wide range of domains, including image and video synthesis, natural language processing, and robotics. Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models, and foundation models transform how agents perceive, learn, and interact with the world. Inspired by this progress, this workshop explores the potential of applying generative modeling techniques to enhance human-robot interaction (HRI). We aim to gather the communities of robot learning and Human-Robot Interaction to discuss cutting-edge generative model techniques, modeling of human behaviors and robot motions, and opportunities to use them to achieve more intuitive human-robot interactions for robot teaching, data collection, language-base interactions and collaborative execution. 

Why are generative models important for research in HRI? HRI will benefit greatly from powerful large models that bring open-world knowledge and generalization to the classic HRI interaction workflows. Just as ChatGPT has become of popular use for non-technical users, it’s only a matter of time before these types of large models with vision and language capabilities will play a key role in generating and mediating interaction between humans and robots in daily life settings (home robots learning your home tasks from examples) and industrial deployments (co-bots in manufacturing). Generative models are also key for the creation of simulation environments (3D assets, scenes, tasks, language commands, and language-based task generation), and simulation environments are useful for data collection of human demonstrations, data generation, and policy training. It’s important for HRI researchers to foster collaborations that investigate how multi-agent interactions and human-like behaviors will play a role in these systems, whether in simulation or real settings. 

Why is HRI important to research in generative models? Conversely, HRI is pivotal for advancing research in generative models. Human interaction and feedback are essential for producing high-quality data for learning and value-aligned training. For example, reinforcement learning from human feedback (RLHF) has demonstrated significant advancements in model performance, enabling ChatGPT’s performance to surpass models learned from static language datasets. Generative models applied to robotics are fundamentally tied to human interaction. In data collection pipelines, we need to provide users with tools, methods, and interfaces to provide and curate high-quality data that can be used by learning algorithms. For model improvements, we need human feedback in the loop with policy learning iterations of fine-tuning during deployment. These are all core interaction problems that are studied in HRI and are now prime to be used in the loop with generative AI in both training and inference, bringing the knowledge from interactions and human-centered modeling into robot learning. 

Link to last year's workshop: https://sites.google.com/view/gai-hri-2024

Motion and Behavior Modelling and Generation 

• Generative modeling of human-like behaviors in imitation learning

• Generative modeling of valid robot motion plans and TAMP

• Generative modeling of human-robot interactions

• Imitation learning and learning from demonstrations (motion and tasks) 

• Imitation of multi-agent collaborative tasks 

• Diffusion Models for motion and behavior generation

• Generation of scenes, tasks, and interactive behaviors in simulation

Human Interaction for Goal Specification on Generative Models

• Teleoperation and shared autonomy

• User goal specification for interactively commanding a robot

• Goal abstractions using language

• Interfaces for robot teaching

• Inference-time policy adaptation using human feedback

Large Language Models (LLMs) and Vision Language Models (VLMs) in HRI

• LLMs and VLMs for embodied AI

• Generative models (LLMs/VLMs) for offline evaluation

• Generative models with multi-modality (vision, language, audio, tactile)

• Generative models of speech for HRI (dialogue, empathy, engagement)

• LLMs as planners for behavior generation

• LLMs and VLMs as reward functions or success detectors

AI-HRI Safety and Alignment 

• Risks and biases of using generative models for data generation, interaction

• Safely deploying generative models for HRI

• Out-of-distribution settings in HRI

• Uncertainty and Misalignment Detection

Authors are invited to submit short papers (3-4 pages excluding references) covering topics on generative modeling applied to human-robot interaction. We invite contributions describing on-going research, results that build on previously presented work, systems, datasets and benchmarks, and papers with demos (that could be displayed easily next to a poster).

Submission link https://openreview.net/group?id=roboticsfoundation.org/RSS/2025/Workshop/GenAI-HRI 

Submission

Submissions should use the official RSS LaTeX template.  Reviews will be single blind. Accepted papers will be presented as posters during the workshop and selected works will have an opportunity for a spotlight talk. Accepted papers will be available online on the workshop website (non-archival). A best paper award will be sponsored by NVIDIA. 

Important Dates 

Submission deadline: May 19th, 2025 (Anywhere on Earth [AoE])

Notifications: May 26th, 2025 (AoE)

Camera-ready deadline: June 1st, 2024 (AoE) - Submit revision in OpenReview + 1min videos submissions. 

Workshop date: June 25th, 2024 (Full day)

Spotlight talks: Accepted papers will be presented as 1min spotlight talks (in-person or pre-recorded videos) in the 3pm session. 

Posters Session: Papers will also present a poster during the poster sessions

Accepted Papers: 

• TBD