2nd Workshop on Scaling Robot Learning

A long-standing goal in robotics is to build generalist robots: a single agent capable of performing a wide variety of everyday tasks based on user instructions. Despite the impressive progress of learning-based methods in robotics, we are far away from achieving robots that can scale to many tasks and to more intuitive task specifications such as videos or natural language.

In this workshop, we are specifically interested in the challenges connected to “scaling” machine learning towards general-purpose robots. Beyond scaling robot systems, this second edition of the workshop aims to focus on how academia can contribute. This includes topics on algorithmic advancements such as self-supervised learning, lifelong learning, multi-task learning, learning structure and priors from large-scale unlabeled multimodal data, learning from videos, and learning in simulation. The goal of this workshop is to bring together experts from different communities, such as robotics, computer vision, natural language processing or reinforcement learning, to discuss challenges, synergies and explore new directions towards scaling robot learning. We also encourage contributions that open-source code, pre-trained models, and datasets.

The workshop will consist of invited talks, spotlight presentations, a poster session and interdisciplinary panel discussions. We aim to improve the communication across a diverse set of scientists who are at various stages of their careers. Instead of the common trade-offs between attracting a wider audience with well-known speakers and enabling early-stage researchers to voice their opinion, we encourage each of our senior presenters to share their presentations with a PhD student or postdoc from their lab. We also ask all our presenters - invited and contributed - to add a “dirty laundry” slide, describing limitations of their work. We expect this will aid further discussion in poster and panel sessions in addition to helping junior researchers avoid similar roadblocks along their path.

The focus topics for our workshop include, but are not restricted to:

• Learning from offline data (pre-trained models, YouTube, etc.)

• Sample efficiency (structure, symmetry, modularity, primitives, etc.)

• Multi-task learning

• Language conditioned learning

• Learning from videos

• Self-supervised learning

• Life-long learning

• Multi-agent systems

Room 750 CEPSR

Gather.town: https://app.gather.town/app/P6V9fyCw1AxuUGWu/Scaling%20Robot%20Learning

YouTube Recordings: https://www.youtube.com/playlist?list=PLkWItR7T--mc3ph1IhKq8wiwpj4kQ3stE

9:00 am - 9:05 am Welcome organizers

9:05 am - 9:30 am Invited Talk, Cewu Lu

9:30 am - 9:55 am Invited Talk, Franziska Meier

9:55 am - 10:30 am Spotlight Sessions (5 min / paper)

1. R3M: A Universal Visual Representation for Robot Manipulation

2. When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning

3. Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding

4. Policy Architectures for Compositional Generalization in Control

5. Jump-Start Reinforcement Learning

6. Skill Induction and Planning with Latent Language

7. Equivariant Q Learning in Spatial Action Spaces

10:30 am - 11:10 am (40 min) Coffee Break & Poster Sessions

11:10 am - 11:35 am Invited Talk, Kristen Grauman

11:35 am - 12:00 pm Invited Talk Marc Toussaint

12:00 pm - 12:30 pm QA + Panel Discussion (Moderator: Pete Florence)

12:30 pm - 2:00 pm (1.5 hrs) Lunch

2:00 pm - 2:25 pm Invited Talk, Aleksandra Faust

2:25 pm - 2:50 pm Invited Talk, Oliver Kroemer

2:50 pm - 3:15 pm Spotlight Sessions (5 min / paper)

1. RRL: Resnet as representation for Reinforcement Learning

2. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances

3. Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?

4. Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube

5. What Makes Representation Learning from Videos Hard for Control?

3:15 pm - 3:55 pm (40 min) Coffee Break & Poster Sessions

3:55 pm - 4:20 pm Invited Talk, Georgia Chalvatzaki

4:20 pm - 4:45 pm Invited Talk, George Konidaris

4:45 pm - 5:15 pm QA + Panel Discussion (Moderator: Pete Florence)

5:15 pm - 5:30 pm Awards & Conclusion

We encourage participants to submit their research in the form of a single PDF. Submissions may be up to 4 pages in length, including figures, excluding references and any supplementary material. Please use the RSS conference template. Accepted papers will be presented in a poster session and selected papers as spotlight talks. All submitted contributions will go through a single blind review process. The contributed papers will be made available on the workshop’s website. However, this does not constitute an archival publication and no formal workshop proceedings will be made available, meaning contributors are free to publish their work in archival journals or conference.

Flexiv Robotics is sponsoring a Best Paper Award of $1,000 to be shared by all authors of the awarded paper.

Submission Website: https://cmt3.research.microsoft.com/RSSWSRL2022

LaTeX Template: https://roboticsconference.org/docs/paper-template-latex.tar.gz