Articulate Robots: Utilizing Language for Robot Learning

Introduction

Language is a cornerstone of human evolution that has played a crucial role in developing communication, socialization, and complex societies. In order to seamlessly integrate robots into our everyday environments, they need to be imbued with a similar ability to comprehend and generate language. Considering the recent advances in large language models and the rapidly increasing hardware resources available to robots, language is evolving to be a modality of choice when developing natural and intuitive interfaces for robot systems. However, to unlock the full potential of language-using robots in our everyday environments, they also need to include their current environment context, as well as (in collaborative settings) the behaviors of their human interaction partners. Modeling such human-robot interactions presents many challenges at the intersection of natural language processing, computer vision, robot control, and machine learning. While major advances have been made in the respective fields, unsolved challenges lie at their intersection, such as how abstract verbal communications can be grounded (situated) in the physical world and aligned with users intent, then transformed into precise robot actions. To this end, we believe that there is an inherent need to address the challenges faced by human-robot interaction systems from a holistic perspective. 

Given the rapid rate of progress in leveraging language for robotics across both industry and academia, we bring together renowned researchers across various disciplines from linguistics, vision, and robotics to advance a more holistic connection between these areas. Seeking to foster interdisciplinary discussion regarding recent advances and the identification of promising areas for further research, our speakers offer unique perspectives across both career stages and research areas. With the addition of solicited contributions and a panel discussion, our workshop intends to encourage community participation and help advance this exciting field of robotics.

Submission

We encourage participants to submit their extended research abstracts in the form of a single PDF. Submissions may be 2-4 pages in length, including figures and excluding references and any supplementary material. Please use the RSS conference template. Accepted abstracts will be presented in a poster session and selected contributions as spotlight talks. Submissions may be on recently published work, upcoming papers, position papers, or work in progress contributions.

Contributions will be made available on the workshop's website. However, no formal workshop proceedings will be made available, meaning contributors are free to publish their work in archival journals or conferences. 

Topics of Interest Include (but are not limited to):

• Language in robot learning

• Multimodal sensor fusion

• Modeling human intentions and desires during HRI

• Simulated agents and robots for in-home interaction with human partners

• Emergent communication between agents and humans

• Language and vision grounding

• Cultural differences in language understanding and usage

• Commonsense and knowledge-based reasoning from language models

Important Dates (Extended Deadline):

• Abstract Submission: May 28th (AoE) June 11th (AoE)

• Author Notification: June 11th June 18th

• Workshop Date: July 14th