Introduction

Reinforcement Learning (RL) has had numerous successes in recent years in solving complex problem domains. However, this progress has been largely limited to domains where a simulator is available or the real environment is quick and easy to access. The organizers have identified a set of 9 challenges that, if solved, will take a big step towards enabling RL agents to be deployed on real-world products and systems.

The goals of this workshop are four-fold:

1. Providing a forum for researchers in academia, industry researchers as well as industry practitioners from diverse backgrounds to discuss the challenges faced in real-world systems.

2. Discuss and prioritize research challenges. As a starting point, a prior work proposes 9 challenges that will take an important step towards realizing RL on real-world products and systems. Determining which challenges are most important, which other challenges should be considered, or whether some of the proposed challenges are perhaps not important are all key discussion points.

3. Discussing problem formulations for various identified challenges and critique these formulations or develop new ones. This is especially important for more abstract challenges such as explainability. We should also be asking ourselves whether the current Markov Decision Process (MDP) formulation is sufficient for solving these problems or whether modifications need to be made.

4. Discuss approaches to solving combinations of these challenges.

One of the major goals of this workshop is to make the research community more aware of the challenges that currently prevent RL agents from working well in real-world systems. We hope that this workshop will encourage more research along each of the challenge dimensions. Especially with respect to formulating or modifying the standard MDP formulation to better model these challenges.

In addition, we hope to bridge communities by providing researchers from diverse backgrounds with more context as to the requirements and constraints of real-world systems, and industry practitioners with more context as to the infrastructure requirements and capabilities of RL agents. We also aim to agree on a prioritization for the various challenges such that there can be an organized effort and synergy between researchers in academia and industry. In addition, the research output will hopefully also be aligned with the requirements of real-world systems as defined by the industry practitioners.

We also hope to promote diversity and ensure that researchers and practitioners from all backgrounds have an equal say in the direction of future research in this field.

Call for Papers

We invite single-blind submissions of 4-8 pages (in NeurIPS or ICLR format) on the following topics:

• Insights in RL inspired by applications.

• Real-world applications of RL to systems in production (industrial, user-facing etc.)

• Discussions on the challenges of real-world RL.

• Alternative approaches to learnt control that extend or even ignore the MDP formulation.

• Theoretical contributions linked to real-world challenges (e.g, safety, robustness, system delays).

• HCI for RL, approaches to teaching new skills to learning systems.

• Advances in explainability of learnt controllers.

Important Dates

Schedule for contributed workshop papers:

• Workshop Date: Saturday Dec 12, 2020

• ***DEADLINE EXTENDED: Submission Date for Workshop Contributions: Oct 09, 2020 (Timezone: 11:59PM PST)

• ***DEADLINE EXTENDED: Acceptance date: Oct 30, 2020

• CMT Submission link: https://cmt3.research.microsoft.com/RWRL2020

Speakers

Schedule (PST Timezone)

• 08:30-08:40 Introduction and Overview

• 08:40-09:20: Keynote: Aviv Tamar: Real World RL Challenges

• 09:20-10:00: Keynote: Emma Brunskill: More practical Batch Offline Reinforcement Learning

• 10:00-10:40 Keynote: Jost Tobias Springenberg: Challenges for RL in Robotics

• 10:40-11:20: Mini-panel discussion 1 - Bridging the gap between theory and practice Aviv Tamar, Emma Brunskill, Jost Tobias Springenberg

• 11:20-11:50: Poster Session 1: Gather Town Link [https://neurips.gather.town/app/uvje3g9tFQHEzyfJ/rwrl]

• 11:50-12:30 Keynote: Franziska Meier: Challenges of Model-based Inverse Reinforcement Learning

• 12:30-13:10: Keynote: Marc Raibert, Scott Kuindersma: Boston Dynamics

• 13:10-13:50 Mini-panel discussion 2 - Real World RL: An industry perspective Marc Raibert, Scott Kuindersma, Franziska Meier

• 13:50-15:20: Lunch Break

• 15:20-16:00: Spotlight Talks

• 16:00-16:40: Keynote: Thomas Dietterich: Applying RL to Ecosystem Management: Lessons Learned

• 16:40-17:20: Keynote: Chelsea Finn: Reinforcement Learning for Real Robots

• 17:20-18:00: Mini-panel discussion 3 - Prioritizing Real World RL Challenges Tom Diettrich, Chelsea Finn, Anca Dragan and Angela Schoellig

• 18:00-18:30: Poster Session 2: Gather Town Link [https://neurips.gather.town/app/uvje3g9tFQHEzyfJ/rwrl]

• 18:30-19:10: Keynote: Angela Schoellig: Machine Learning for Safety-Critical Robotics Applications

• 19:10-19:50: Keynote: Anca Dragan: Reinforcement Learning that optimizes what people really want

Organizers

Spotlight Papers

Accepted Papers

Program Committee