Behaviour Priors in Reinforcement Learning for Robotics Workshop

IEEE International Conference on Robotics and Automation (ICRA) 2022

May 27, 2022 Philadelphia, USA

ROOM 119 B


The use of prior knowledge in reinforcement learning (RL) has become ubiquitous as we move towards developing systems suitable for real-world robotics. Recent works have shown a resurgence of interest in methods that incorporate prior knowledge in the form of behaviour priors. Behaviour priors are systems that can propose potentially meaningful behaviours for an agent to take when in a given state, including classical handcrafted controllers, previously learned policies, demonstrations and more recently latent skills modules. Such systems have proven their effectiveness in RL for accelerated training, safer exploration and generalisation across a wide range of tasks.

The goal of this workshop is to bring together researchers across a variety of domains, including RL, robotics and control, to discuss the role that behaviour priors could play in RL. This includes the various ways in which we can learn/model these priors, methods to integrate their experience within the RL framework and their applicability to solving some of the key challenges faced by RL for real-world robotics.

Invited Speakers

Nicolas Heess


Doina Precup

McGill University

Davide Scaramuzza

University of Zurich

Avi Singh

Google Brain

Animesh Garg

University of Toronto

Kelsey Allen


Panel Moderator

Dushyant Rao


Call for Papers

We invite the submission on topics including, but not limited to:

  • Skills learning of behaviour priors

  • Behaviour priors in hierarchical RL

  • Meta-learning behaviour priors

  • Behaviour priors for planning

  • Modularity and compositionally of behaviour priors

  • Residual RL

  • Continual/transfer learning in RL

  • Exploration bias using behaviour priors

  • Implications of information asymmetry

  • Dynamic Movement Primitives as behaviour priors

We also invite abstracts that address the following questions directly:

  1. What is the trade-off between generality and the use of behaviour priors in RL, in the context of specific tasks or in general, and how can we evaluate this in practice?

  2. What are the practical or theoretical implications of specific ways of imposing or learning behaviour priors in RL?

  3. How can we learn data-driven behaviour priors for RL (via latent space models, meta-RL, transfer learning, multi-task RL)?

  4. What structure and properties should the behaviour priors exhibit to be general and transferable, and could they be learned?

  5. How can we leverage existing controllers from the robotics community in RL for safe and efficient learning as well as across multiple tasks?

  6. How can the different communities (including control and robotics) benefit from collaborative research on these topics?

The submission deadline is 1 May 2022, and decisions will be sent out on 7 May 2022. The goal of this workshop is to bring together researchers across a variety of domains, including RL and machine learning practitioners, and roboticists to discuss the role that behaviour priors could play in RL.

Please submit papers via CMT here:

Submissions should be in the IEEE ICRA format with a maximum of 4 pages, not including references. Authors may submit up to 100MB of supplementary material, such as appendices, proofs, derivations or data; all supplementary materials must be submitted as a separate file in PDF or ZIP format.

Accepted submissions will be presented in the form of posters or contributed talks.

Submissions to the workshop cannot have been accepted as conference papers at ICRA (or other conferences). It is okay for submissions to be under review elsewhere.

Important Dates and Deadlines

For any questions or clarification feel free to email the organising committee at:


  • 08:30 - 08:45 Welcome and Introduction

  • 08:45 - 09:15 Invited Speaker: Nicolas Heess

Behaviour priors: From Skills to Tasks

  • 09:15 - 09:45 Invited Speaker: Doina Precup

What can Hierarchical RL Do for You?

  • 09:45 - 10:15 Invited Speaker: Davide Scaramuzza

Learning to Fly with Agility

  • 10:15 - 11:00 Coffee Break

  • 11:00 - 11:30 Invited Speaker: Avi Singh

Data-Driven Behaviour Priors for Reinforcement Learning

  • 11:30 - 12:00 Invited Speaker: Animesh Garg

Structured Inductive Bias for Robot Learning

  • 12:00 - 13:30 Lunch Break

  • 13:30 - 14:00 Invited Speaker: Kelsey Allen

Towards more human-like, structured behavioural priors for tool use and construction.

  • 14:00 - 14:45 Panel Discussion: Dushyant Rao

(Moderator) + Invited Speakers

  • 14:45 - 15:15 Spotlight talks

    • 14.45 - 14.50 Snehal Jauhri

Robot Learning of Mobile Manipulation with Reachability Behaviour Priors

  • 14.50 - 14.55 Yunlong Song

Policy Search for Model Predictive Control with Application to Agile Drone Flight

  • 14.55 - 15.00 Niklas Funk

Graph-based Reinforcement Learning meets Mixed Integer Programs: An application to 3D robot assembly discovery

  • 15.00 - 15.05 Shivansh Beohar

Planning with RL and Episodic-Memory Behaviour Priors

  • 15.05 - 15.10 Alberto Giammarino

A Teleoperation Framework for Impedance-based Behavior Priors in RL for Enhancing Contact-rich Tasks: A Design Concept

  • 15:15 - 15:20 Final Remarks and Conclusion

Accepted Papers


Krishan Rana

QUT Centre for Robotics

Andrew Melnik

University of Bielefeld

Kate Rakelly

UC Berkeley

Niko Suenderhauf

QUT Centre for Robotics

Programme Committee

Jordan Erskine

Robert Lee

Brendan Tidd

Fangyi Zhang

Augustin Harter

Christian Limberg

Recorded Talks

Krishan Rana: Introduction | What are Behaviour Priors?

Nicolas Heess: From Skills to Tasks

Doina Precup: What can Hierarchical RL Do for You?

Avi Singh: Data-Driven Behaviour Priors for RL

Davide Scaramuzza: Learning Agile Vision-Based Flight

Animesh Garg: Structure in RL for Robotics

Kelsey Allen: Behaviour Priors for Tool Use and Construction

Panel Discussions

Spotlight Talks