Workshop on Combining Learning and Motion Planning

September 27, 16:10 CEST (7:10 PST, 22:10 CST)

IROS 2021


The workshop will be held virtually on the IROS 2021 conference platform (Hall 3)

Invited Talks

The recorded talks are available on the IROS conference system

After you login, click on Program -> Program Overview -> Search for 'CLAMP: Combining Learning and Motion Planning for Robotics'

Self-supervision in Motion Planning

Aleksandra Faust

Google Brain

Toward Fully-Automated Learning of Model Predictive Control

Kris Hauser


Rethinking Representations for Robotics

Lerrel Pinto

New York University

Learning Decentralized Motion Planners for Multi-Robot System

Shuran Song

Columbia University

Learning from Planning

Marc Toussaint

TU Berlin


Motion planning, one of the cornerstones of robotics, can generate spatial trajectories to accomplish navigation and manipulation tasks with theoretical optimality guarantees. However, motion planners require accurate forward models both for the robot and environment, comprehensive geometric models for collision checking, and a significant amount of computation required to cover the entire space of possible motion actions.

On the other hand, machine learning approaches provide alternative ways to generate trajectories without the need for access to environment models. They are able to learn a mapping from noisy high-dimensional sensory inputs (e.g. images) to the right actions. However, they often require a large amount of training data, which is difficult to collect on real robot systems.

In recent years, these two approaches, learning and motion planning, have been combined in various ways to achieve the advantages of both of them. For example, a motion planner can be integrated into the policy of a learning agent; a learned dynamics model can be used for motion planning; traditional motion planners can be distilled into a neural network that learns a mapping from images to possible trajectories; and finally, sampling strategy for motion planning can be learned to improve exploration. The possibilities of combining machine learning and motion planning synergistically seem to not have been yet depleted!


The goal of this workshop is to provide a platform for roboticists and machine learning researchers to discuss the past achievements of motion planning and learning as well as future research directions of this research area. We would like to cover past and future ways of integrating these two worlds to overcome their individual limitations. We invited a group of speakers that are world renowned experts to present their work on combining learning and motion planning. We have also planned guided panel discussions to encourage debate among the invited speakers and workshop participants on the questions:

  • How can motion planning improve sample efficiency of robot learning approaches?

  • How to incorporate learned models (e.g. perception, dynamics) in motion planning?

  • Can learning techniques help to transfer motion plans from simulation to real robots?

  • How to efficiently switch between learned and planned behaviors?

  • Can learning approaches help with the partial observability problem?

  • In which situations to use motion planning and when to use learning?


  • Reinforcement learning and imitation learning with motion planning

  • Learning models (e.g., dynamics, perception) for planning

  • Robotic manipulation and navigation in obstructed environments

  • Planning for mobile manipulation

  • Simulation to real world transfer for planning or with planning

  • Distilling motion planner into deep neural networks (a.k.a. neural motion planner)

  • Benchmark and task proposals

  • Software for research and deployment of robot learning and motion planning algorithms