Objectives

In order for robots to be able to perform a wide variety of tasks in human environments, they need to be able to combine actions in intelligent ways to accomplish long-horizon, unseen tasks, while communicating their intentions and capabilities to the humans with whom they share an environment. To enable this capability, robots must be endowed with some knowledge of these actions, and an ability to reason about their consequences: both key elements of robot cognitive behavior.

In robotics research today, we generally view actions in two ways: first, as control policies responding to low-level sensor data; and second, as high-level symbolic actions. Action semantics can bridge these two levels, informing not just what to do but how to do it, and enabling effective human-robot collaboration in addition to autonomy.

Drawing inspiration from recent advances in large-scale, general representation learning in computer vision and natural language processing, we feel that learned, general purpose action semantics for robotics are on our immediate horizon. Semantic representation provides a tool for capturing the essence and function of actions, helping robots learn and generalize across task and motion planning domains. More descriptive, learned semantic action representations will yield robots with greater capability and autonomy in a wide range of human environments.

Because learned action semantics are an important and growing space, we want to use this workshop to highlight the research and voices of junior researchers. As a result, we are committing to having at least half of our invited speakers be final-year PhD students, postdocs, or first year faculty with relevant perspectives.

In this full-day workshop, we aim to answer two major questions:

How can we learn scalable and general semantic representations? In recent years, there has been a substantial contribution in semantic policy and action representation in the fields of robotics, computer vision, and machine learning. In this respect, we would like to invite experts in academia and motivate them to comment on the recent advances in semantic reasoning by addressing the problem of linking continuous sensory experiences and symbolic constructions to couple perception and execution of actions. In particular, we want to explore how these can make robot learning more scalable and generalizable to new tasks and environments.

How can semantic information be used to create Explainable AI? We would like to invite researchers from a broad range of areas including task and motion planning, language learning, general-purpose machine learning, and human-robot interaction. Much of action semantics is definitionally tied to how robots and humans communicate, and one fundamental feature of these approaches should be that they allow a broad variety of people to benefit from advances in robotics, and to work alongside robots outside of laboratory environments. Building more understandable action representations is important as a way of building robotic systems that benefit society.

Important Dates

• Workshops paper submission deadline (extended): September 3, 2021. August 6, 2021

• Workshops paper notification (extended): September 13, 2021. September 3, 2021

• Workshops camera-ready submission (extended): September 20, 2021. September 15, 2021

• Workshop date: September 27, 2021.

Also check out the associated Robotics and Autonomous Systems journal special issue.