RSS 2019 Workshop on

Combining Learning and Reasoning – Towards Human-Level Robot Intelligence

June 22 (Saturday) 2019, Freiburg im Breisgau, Germany

Location: University of Freiburg, Faculty of Engineering, Building 101, Room 016-018


Robotics research has developed powerful model-based methods for perception, state estimation, planning, control, etc., which form the building blocks of the vast majority of successful robot systems. At the same time, data-driven, model-free learning has recently brought unprecedented success in various domains, where model-based methods struggle despite decades of research. The aim of this workshop is to bring together researchers from robotics and machine learning, and discuss opportunities and challenges towards building human-level robot intelligence, in particular, combining model-based reasoning and data-driven learning in a scalable and composable manner.

Some questions we would like to discuss:

  • What is the role of model-based reasoning and model-free learning in an intelligent robot system?
  • Can we learn intelligent robot behaviour only from reinforcements? Is expert knowledge essential?
  • How do we combine existing knowledge with data-driven learning? What should be built in and what should be learned?
  • How do we integrate solutions for small, isolated sub-tasks into a large intelligent system? How do we combine model-based and model-free components? Would we build Shakey differently today?
  • Is robot intelligence going to be explainable? Is interpretability unnecessary, good to have, or a must?

The workshop will consist of invited talks, mini-debates, contributed talks, spotlight talks, and two poster sessions.


  • 08:00 - 09:00: Registration

  • 09:00 - 09:15: Opening remarks
  • 09:15 - 09:50: Invited talk. Raia Hadsell (DeepMind): In defense of Model-free Robots.
  • 09:50 - 10:05: Contributed talk.
    • Gilwoo Lee: Residual Bayesian Q-Learning for Meta-Reinforcement Learning with Experts.
  • 10:05 - 10:15: Spotlight presentations: Number 1 - 5

  • 10:15 - 10:45: Coffee break

  • 10:45 - 11:20: Invited talk. David Hsu (NUS): Algorithm + Neural Network = ?
  • 11:20 - 11:55: Invited talk. Alberto Rodriguez (MIT): A Hierarchy of Manipulation Planning Problems to solve and Decisions to make.
  • 11:55 - 12:15: Spotlight presentations: Number 6 - 12

  • 12:15 - 13:00: Poster session #1
  • 13:00 - 14:00: Lunch break

  • 14:00 - 14:35: Invited talk. Byron Boots (Georgia Tech): Online Learning for Adaptive Robotic Systems.
  • 14:35 - 15:00: Contributed talks.
    • Georg Martius: Control What You Can Intrinsically Motivated Task-Planning Agent.
    • Brian Hou: Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributions.

  • 15:00 - 15:30: Coffee break

  • 15:30 - 16:05: Invited talk. Nicolas Heess (DeepMind): TBD.
  • 16:05 - 16:40: Invited talk. Marc Toussaint (University of Stuttgart): On Combining Learning & Reasoning
  • 16:40 - 16:45: Closing remarks

  • 16:45 - 17:30: Poster session #2


Faculty of Engineering, Building 101 (same as registration), Room 016-018.

Venue information:

Call for Participation

We invite participants from all fields of robotics to submit extended abstracts related to the workshop topic, anywhere on the spectrum between model-free learning and model-based reasoning. Submissions should be up to 2-4 pages + references using the RSS template and should not be anonymized.

Submissions can include original research, late breaking results, position papers, and literature reviews. Significant overlap with work submitted to other venues is acceptable, as long as it has not yet been published or presented.

Accepted submissions will be presented in 2-minute spotlight talks and in a dedicated poster session. Selected contributions may receive a longer presentation slot. All accepted abstracts will be posted on the workshop website.

  • Submission deadline: June 5, 2019 (AoE)
  • Notification of acceptance: June 8, 2019
  • Camera-ready deadline: June 21, 2019 (1pm GMT)

Submission website:

Invited Speakers

  • Alberto Rodriguez, Massachusetts Institute of Technology
  • Raia Hadsell, Google DeepMind
  • David Hsu, National University of Singapore
  • Nicolas Heess, Google DeepMind
  • Byron Boots, Georgia Tech
  • Marc Toussaint, University of Stuttgart

Accepted Submissions

Contributed Talks

Poster Presentation

  1. Markus Merklinger, Oier Mees, Gabriel Kalweit and Wolfram Burgard: Adversarial Skill Networks: Unsupervised Skill Learning from Video
  2. Gabriel Kalweit, Maria Huegle and Joschka Boedecker: Composite Q-Learning
  3. Bernadette Bucher, Anton Arapin, Ramanan Sekar, Feifei Duan, Marc Badger, Kostas Daniilidis and Oleh Rybkin: Perception-Driven Curiosity with Bayesian Surprise
  4. Adam Allevato, Elaine Short, Mitch Pryor and Andrea Thomaz: Learning A Human-Centered Representation of Robot Affordance Models
  5. Mohan Sridharan: Refinement-Based Architecture for Knowledge Representation, Explainable Reasoning and Interactive Learning in Robotics
  6. Niko Grupen and Ross Knepper: Visual Primitives for Abductive Reasoning
  7. Natasha Danas, Cobi Finkelstein and Stefanie Tellex: Formal Dialogue Model for Language Grounding Error Recovery
  8. Nakul Gopalan, Eric Rosen and Stefanie Tellex: Towards Mapping Language to Portable Symbols for Instruction Following
  9. Brian Hou, Sanjiban Choudhury, Gilwoo Lee, Matt Barnes and Siddhartha Srinivasa: Collision Posteriors on Graphs with Expensive-to-Evaluate Edges
  10. Ankit Shah and Julie Shah: Planning with Uncertain Specifications
  11. Zlatan Ajanovic, Halil Beglerovic and Bakir Lacevic: A novel approach to model exploration for value function learning
  12. Panpan Cai, Yuanfu Luo, Aseem Saxena, David Hsu and Wee Sun Lee: LeTS-Drive: Driving in a Crowd by Learning from Tree Search


  • Peter Karkus, National University of Singapore
  • Alina Kloss, Max Planck Institute for Intelligent Systems
  • Rico Jonschkowski, Robotics at Google
  • Leslie P. Kaelbling, Massachusetts Institute of Technology