2nd Workshop on Scaling Robot Learning
RSS 2022 Workshop | June 27, 2022
A long-standing goal in robotics is to build generalist robots: a single agent capable of performing a wide variety of everyday tasks based on user instructions. Despite the impressive progress of learning-based methods in robotics, we are far away from achieving robots that can scale to many tasks and to more intuitive task specifications such as videos or natural language.
In this workshop, we are specifically interested in the challenges connected to “scaling” machine learning towards general-purpose robots. Beyond scaling robot systems, this second edition of the workshop aims to focus on how academia can contribute. This includes topics on algorithmic advancements such as self-supervised learning, lifelong learning, multi-task learning, learning structure and priors from large-scale unlabeled multimodal data, learning from videos, and learning in simulation. The goal of this workshop is to bring together experts from different communities, such as robotics, computer vision, natural language processing or reinforcement learning, to discuss challenges, synergies and explore new directions towards scaling robot learning. We also encourage contributions that open-source code, pre-trained models, and datasets.
The workshop will consist of invited talks, spotlight presentations, a poster session and interdisciplinary panel discussions. We aim to improve the communication across a diverse set of scientists who are at various stages of their careers. Instead of the common trade-offs between attracting a wider audience with well-known speakers and enabling early-stage researchers to voice their opinion, we encourage each of our senior presenters to share their presentations with a PhD student or postdoc from their lab. We also ask all our presenters - invited and contributed - to add a “dirty laundry” slide, describing limitations of their work. We expect this will aid further discussion in poster and panel sessions in addition to helping junior researchers avoid similar roadblocks along their path.
The focus topics for our workshop include, but are not restricted to:
Learning from offline data (pre-trained models, YouTube, etc.)
Sample efficiency (structure, symmetry, modularity, primitives, etc.)
Language conditioned learning
Learning from videos
Room 750 CEPSR
YouTube Recordings: https://www.youtube.com/playlist?list=PLkWItR7T--mc3ph1IhKq8wiwpj4kQ3stE
9:00 am - 9:05 am Welcome organizers
9:05 am - 9:30 am Invited Talk, Cewu Lu
9:30 am - 9:55 am Invited Talk, Franziska Meier
9:55 am - 10:30 am Spotlight Sessions (5 min / paper)
R3M: A Universal Visual Representation for Robot Manipulation
When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning
Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding
Policy Architectures for Compositional Generalization in Control
Jump-Start Reinforcement Learning
Skill Induction and Planning with Latent Language
Equivariant Q Learning in Spatial Action Spaces
10:30 am - 11:10 am (40 min) Coffee Break & Poster Sessions
11:10 am - 11:35 am Invited Talk, Kristen Grauman
11:35 am - 12:00 pm Invited Talk Marc Toussaint
12:00 pm - 12:30 pm QA + Panel Discussion (Moderator: Pete Florence)
12:30 pm - 2:00 pm (1.5 hrs) Lunch
2:00 pm - 2:25 pm Invited Talk, Aleksandra Faust
2:25 pm - 2:50 pm Invited Talk, Oliver Kroemer
2:50 pm - 3:15 pm Spotlight Sessions (5 min / paper)
RRL: Resnet as representation for Reinforcement Learning
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
What Makes Representation Learning from Videos Hard for Control?
3:15 pm - 3:55 pm (40 min) Coffee Break & Poster Sessions
3:55 pm - 4:20 pm Invited Talk, Georgia Chalvatzaki
4:20 pm - 4:45 pm Invited Talk, George Konidaris
4:45 pm - 5:15 pm QA + Panel Discussion (Moderator: Pete Florence)
5:15 pm - 5:30 pm Awards & Conclusion
Thanks for Participating!
SAGCI-System: Towards Sample-Efficient, Generalizable, Compositional, and Incremental Robot Learning [paper]
R3M: A Universal Visual Representation for Robot Manipulation [paper]
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances [paper]
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation? [paper]
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube [paper]
When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning [paper]
Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding [paper]
Correct and Certify: A New Approach to Self-Supervised 3D Object Perception [paper]
Policy Architectures for Compositional Generalization in Control [paper]
Jump-Start Reinforcement Learning [paper]
Skill Induction and Planning with Latent Language [paper]
Equivariant Q Learning in Spatial Action Spaces [paper]
Scaling Multi-Agent Reinforcement Learning via State Upsampling [paper]
What Makes Representation Learning from Videos Hard for Control? [paper]
RRL: Resnet as representation for Reinforcement Learning [paper]
Towards Language-Conditioned Observation Models for Visual Object Search [paper]
Sim2Real on the Robotarium Platform Using Decentralized Multi-Agent Safe Deep FBSDEs [paper]
Learning Recovery from Failures on a Budget for Robust Manipulation [paper]
For printing physical posters, please refer to the RSS format guidelines https://roboticsconference.org/attending/presenters/
To accommodate multiple timezones in this hybrid workshop format authors should, if feasible, try and attend both sheduled sessions to present their poster. If not possible, please present at the session that best works for you.
Onsite authors should bring their webcam-enabled personal laptops to connect to gather.town for a hybrid poster session.
Call for Contributions
We encourage participants to submit their research in the form of a single PDF. Submissions may be up to 4 pages in length, including figures, excluding references and any supplementary material. Please use the RSS conference template. Accepted papers will be presented in a poster session and selected papers as spotlight talks. All submitted contributions will go through a single blind review process. The contributed papers will be made available on the workshop’s website. However, this does not constitute an archival publication and no formal workshop proceedings will be made available, meaning contributors are free to publish their work in archival journals or conference.
Flexiv Robotics is sponsoring a Best Paper Award of $1,000 to be shared by all authors of the awarded paper.
Submission Website: https://cmt3.research.microsoft.com/RSSWSRL2022
LaTeX Template: https://roboticsconference.org/docs/paper-template-latex.tar.gz
15, 2022 Submission Deadline (AoE time)
June 12, 2022 Author Notification
June 19, 2022 Camera Ready Submission
June 27, 2022 Workshop
For further information please contact us at rss22-srl(at)googlegroups.com