2021 RSS Workshop on

Advancing Artificial Intelligence and Manipulation for Robotics: Understanding Gaps, Industry and Academic Perspectives, and Community Building
July 13, 2021

Advances in machine learning (ML), the design of novel end-effectors and sensors, and the development of intelligent perception, planning and control algorithms have resulted in undeniable progress in robot manipulation. Impressive demonstrations of multi-fingered dexterity in research labs show that robots are increasingly capable of manipulating the physical world around them. Despite this considerable progress, however, current robotic manipulation skills remain far from human-level versatility. Moreover, academic progress in machine learning and manipulation has not translated into industry; industrial manufacturing and manipulation systems instead operate in highly structured environments and rely on simple perception, planning, and control.

In this workshop, we will bring together experts from academia and industry to identify industrial manufacturing and manipulation problems that can most benefit both from recent academic progress and advances in artificial intelligence, and in order to determine meaningful research directions that are motivated by these applications. In particular, the goals of the workshop are:

  • Determining important criteria for industrial adoption and the gap between these criteria and the current approaches to manipulation in academia.

  • Raise awareness of the need for ML metrics, evaluations, and benchmarks for manufacturing-relevant parts, operations, and environments requiring robots.

  • Convene stakeholders to define common language for discussing ML performance, characteristics, applicability and/or tools and measurement science necessary to advance the state of ML in manufacturing robotics and reduce the risk of adopting ML-based technologies.

  • Form an ongoing community to develop, review, test, mature, and contribute to the concepts and tools that can help advance the field and foster well-informed, successful adoption and implementation of ML-based manufacturing robotics capabilities.

The workshop will host talks by invited speakers, peer-reviewed poster sessions, panel discussions, and open discussions. In particular, we will include talks from representatives in different industries, including manufacturing, warehouse automation, automotive, electronics assembly, and homecare, to better understand their diverse requirements and needs. We expect insightful discussions between experts from industry and academia, leading to a better understanding of avenues for potential collaboration between the two communities.

Please join the discussion after the workshop in the AI for Manufacturing Robotics Community Slack: https://tinyurl.com/ai-mnfg-robotics

And if you're interested in the AI for Manufacturing Robotics community, there is more information on the following website: https://sites.google.com/view/ai4manufacturingrobotics/home


Juan Aparicio Ojea

READY Robotics

Al Rizzi

Boston Dynamics


10:00 - 10:15 (PDT)

13:00 - 13:15 (EDT)

19:00 - 19:15 (CEST)

Welcome and Introductions

10:15 - 10:45 (PDT)

13:15 - 13:45 (EDT)

19:15 - 19:45 (CEST)

"Advanced Robotics for Manufacturing Institute - AI, Enhancing the Performance, Proficiency and Efficiency of Robots Used in Manufacturing"

Arnold Kravitz (ARM)

Video Link

10:45 - 11:10 (PDT)

13:45 - 14:10 (EDT)

19:45 - 20:10 (CEST)

"Roadblocks to Deploy AI-Based Robotic Applications in Manufacturing"

Juan Aparicio Ojea (READY Robotics)

Video Link

11:10 - 11:40 (PDT)

14:10 - 14:40 (EDT)

20:10 - 20:40 (CEST)

Discussion with Session I speakers

11:40 - 12:05 (PDT)

14:40 - 15:05 (EDT)

20:40 - 21:05 (CEST)

"Representation Learning for Interaction Tasks"

Danica Kragic (KTH)

Video Link

12:05 - 12:30 (PDT)

15:05 - 15:30 (EDT)

21:05 - 21:30 (CEST)

"Grasping and Manipulation Opportunities in Automotive Manufacturing"

Parag Patre (Magna)

Video Link

12:30 - 12:55 (PDT)

15:30 - 15:55 (EDT)

21:30 - 21:55 (CEST)

"Design Optimization, Sensor Development, and Dexterous Manipulation Learning For Multifingered Hands"

Matei Ciocarlie (Columbia University)

Video Link

12:55 - 13:25 (PDT)

15:55 - 16:25 (EDT)

21:55 - 22:25 (CEST)

Discussion with Session II speakers

14:00 - 15:00 (PDT)

17:00 - 18:00 (EDT)

23:00 - 00:00 (CEST)

Poster Session

15:00 - 15:25 (PDT)

18:00 - 18:25 (EDT)

00:00 - 00:25 (CEST, July 14)

"Supervised Local Autonomy for Mobile Manipulation with Spot"

Al Rizzi (Boston Dynamics)

Video Link

15:25 - 15:50 (PDT)

18:25 - 18:50 (EDT)

00:25 - 00:50 (CEST, July 14)

"Online Recovery from Failure"

Jeannette Bohg (Stanford)

Video Link

15:50 - 16:15 (PDT)

18:50 - 19:15 (EDT)

00:50 - 01:15 (CEST, July 14)

"Rare events, dynamics, and fleet learning"

Russ Tedrake (MIT)

Video Link

16:15 - 16:45 (PDT)

19:15 - 19:45 (EDT)

01:15 - 01:45 (CEST, July 14)

Discussion with Session III speakers

16:45 - 17:45 (PDT)

19:45 - 20:45 (EDT)

01:45 - 02:45 (CEST, July 14)

Panel discussion

17:45 - 18:00 (PDT)

20:45 - 21:00 (EDT)

02:45 - 03:00 (CEST, July 14)

Closing remarks


Clemens Eppner (NVIDIA)

Neel Doshi (MIT)

Megan Zimmerman (NIST)

Diego Romeres (MERL)

Craig Schlenoff (NIST)

Siyuan Dong (UW)

Devesh Jha (MERL)

Prof. Alberto Rodriguez (MIT)

Michel Breyer (ETH Zürich)

Coline Devin (Deepmind)

Arsalan Mousavian (NVIDIA)

Jingyi Xu (TU München)

Andy Zeng (Google AI)

Holly Yanco (UML)

Adam Norton (UML)

Vinh Nguyen (NIST)

Siddarth Jain (MERL)

Contact Information

If you have any questions or require any further information feel free to contact Megan Zimmerman at megan.zimmerman@nist.gov or one of the other organizers.