Learning and Inference in Robotics:

Integrating Structure, Priors and Models

Robotics: Science and Systems workshop on June 29, 2018


Date: June 29, 2018

Time: 8:55am - 5:00pm

Location: Doherty Hall 2210, CMU, Pittsburgh, USA

We will be using Pigenhole for live and interactive Q&A, go to


and use the following format to ask questions

@<speaker>, <question>


Recent advances in machine learning techniques from the emergence of deep learning, and access to large amounts of data and powerful computing hardware have led to great strides in the state-of-the-art in robotics and artificial intelligence. In contrast to traditional approaches that are strongly model-based with priors and explicit structural constraints, these newer approaches tend to be data-driven and often neglect the underlying problem structure. As a consequence, while these approaches usually outperform their traditional counterparts on many robotics problems, achieving good generalization, task transfer and data-efficiency has been challenging. Combining the strengths of the two paradigms:

  1. The flexibility of modern learning and inference techniques, and
  2. The domain knowledge and structural priors of traditional methods

should help bridge this gap.

The goal of this workshop is to bring together researchers from robotics and machine learning to investigate, at the intersection of the two paradigms, techniques for structured learning and inference. Our notion of "structure" is very general. In the context of robot learning and inference this manifests in many ways: as a specific architecture of a (probabilistic) graphical model or (deep) network, an intermediate representation, a loss function, and so on. Some of the questions we hope to answer include:

  • How can we leverage structure to improve the state of the art in learning/inference models?
  • What is the right mix between structure/priors/models and learning?
  • How can we establish benchmarks/baselines that show the effectiveness of using structure in learning/inference for robotics?

A special emphasis will be on methods that tightly integrate structure with learning and are demonstrably applicable in the real-world, particularly on problems like autonomous navigation, manipulation, and field robotics.

Invited Speakers

Drew Bagnell

Carnegie Mellon University / Aurora

Jeannette Bohg

Stanford University

Angela Schoellig

University of Toronto

Marc Toussaint

Uni Stuttgart / MIT

Kostas Daniilidis

University of Pennsylvania

Zico Kolter

Carnegie Mellon University / Bosch Center for AI (BCAI)

Important Dates

May 19

May 26

June 7

June 14

June 29

Submission deadline (AoE time)

Extended submission deadline (AoE time)

Notification of acceptance

Camera ready deadline


Call for Abstracts

We solicit up to 3 pages extended abstracts (excluding citations) conforming to the official RSS style guidelines. Submissions can include archived or previously accepted work (please make a note of this in the submission). Reviewing will be single blind.

Submission link: https://easychair.org/conferences/?conf=rss18lair

Topics of interest include, but are not limited to:

  • Structured inference and learning for robotics
  • Deep learning with structure and priors
  • Learning structured representations for perception, planning and control
  • Integrating learning and model-based robotics
  • Structured losses and semi/self-supervised learning
  • Reinforcement/Imitation learning using domain knowledge
  • Autonomous navigation, mobile manipulation with structured learning
  • Structured optimization with deep learning and automatic differentiation
  • Deep learning with graphical models

All accepted contributions will be presented in interactive poster sessions. A subset of accepted contributions will be featured in the workshop as spotlight presentations.


Mustafa Mukadam

Georgia Institute of Technology

Arunkumar Byravan

University of Washington

Byron Boots

Georgia Institute of Technology