Generalization in Planning

S1 - 8:30am - 10:30am (Session Chair: Javier Segovia-Aguas)

• Workshop opening (5 minutes)
• Leslie Kaelbling - Policies, Models and Q: Oh My!
• Peter Stone - Task-Motion Planning with Reinforcement Learning for Adaptable Mobile Service Robots
• Spotlight talks (25 minutes):
  1. Sarah Keren, Luis Pineda, Avigdor Gal, Erez Karpas and Shlomo Zilberstein - Efficient Heuristic Search for Optimal Environment Redesign.
  2. Sarah Keren, Haifeng Xu, Kofi Kwapong, David Parkes and Barbara Grosz - Information Shaping for Enhanced Goal Recognition of Partially-Informed Agents.
  3. Philip Korsunsky, Stav Belogolovsky, Tom Zahavy, Chen Tessler and Shie Mannor - Inverse Reinforcement Learning in Contextual MDPs.
  4. Huazhe Xu, Boyuan Chen, Yang Gao and Trevor Darrell - Scoring-Aggregating-Planning: Learning task-agnostic priors from interactions and sparse rewards for zero-shot generalization.
  5. William Shen, Felipe Trevizan and Sylvie Thiebaux - Learning Delete-Relaxation Heuristics over Hypergraphs.
  6. Richard Freedman, Scott Friedman, David Musliner and Michael Pelican - Creative Problem Solving Through Automated Planning and Analogy.
• Poster Session (15 minutes + breaks for discussion)

Break 1 - 10:30am - 11:00am (poster discussion)

S2 - 11:00am - 12:30pm (Session Chair: Christian Muise)

• Sheila McIlraith - Learning and Remembering: Planning with and without instruction
• Long paper talks (60 minutes):
  1. León Illanes and Sheila McIlraith - Generalized Planning via Abstraction: Arbitrary Numbers of Objects. (slides)
  2. Guillem Francès , Augusto B. Corrêa , Cedric Geissmann and Florian Pommerening - Generalized Potential Heuristics for Classical Planning.
  3. Jeevana Priya Inala, Osbert Bastani, Zenna Tavares and Armando Solar-Lezama - Synthesizing Programmatic Policies that Inductively Generalize. (ICLR version)
  4. Tom Silver, Rohan Chitnis, Anurag Ajay, Josh Tenenbaum and Leslie Kaelbling - Learning Skill Hierarchies from Predicate Descriptions and Self-Supervision.
  5. Alejandro Suárez-Hernández, Javier Segovia-Aguas, Carme Torras and Guillem Alenya - STRIPS Action Discovery.
  6. Pulkit Verma and Sidddharth Srivastava - Learning Generalized Models by Interrogating Black-Box Autonomous Agents.

Lunch - 12:30am - 2:00pm

S3 - 2:00pm - 3:30pm (Session Chair: Tomas Lozano-Perez)

• Hector Geffner - Representation, learning, and planning
• Long paper talks (40 minutes):
  1. Yi Wu, Yuxin Wu, Aviv Tamar, Stuart Russell, Georgia Gkioxari and Yuandong Tian - Bayesian Relational Memory for Planning in Semantic Visual Navigation.
  2. Richa Verma, Aniruddha Singhal, Harshad Khadilkar, Ansuma Basumatary, Siddharth Nayak, Harsh Vardhan Singh, Swagat Kumar and Rajesh Sinha - A Generalized Reinforcement Learning Algorithm for Online 3D Bin-Packing.
  3. Frederick Callaway, Bas van Opheusden, Erin Grant and Thomas Griffiths - Generalization in planning by metalearning to metareason.
  4. Kara Liu, Thanard Kurutach, Aviv Tamar and Pieter Abbeel - Hallucinative Topological Memory for Zero-Shot Visual Planning. (ICLR version)

Break 2 - 3:30pm - 4:00pm (poster discussion)

S4 - 4:00pm - 5:30pm (Session Chair: Siddharth Srivastava)

• Giuseppe De Giacomo - Reactive Synthesis + World Model = Planning
• George Konidaris - Learning Portable Skills and Symbols
• Panel discussion (40 minutes with invited speakers)

These talks consist of 45 minutes presentations and 5 minutes for questions.

Leslie Kaelbling

Policies, Models and Q: Oh My!

There is a lot of disagreement in the world of adaptive intelligent systems about the relative merits of "model-free" and "model-based" learning methods. It seems clear to me that they both have important roles to play in the design of flexible, intelligent robots, and we should try to understand them in terms of generalization capacity, sample complexity, and both on-line and off-line computation requirements. I'll discuss strategies for combining policy learning, model learning, value-function learning, and planning with the goal of designing systems that exhibit lifelong, incremental learning with combinatorial generalization.

Hector Geffner

Representation, learning, and planning

Generalized planning (GP) is the model-based approach to the computation of general plans; i.e. plans that solve multiple planning problems, in certain cases, infinitely many problem instances (e.g. general plans that solve any block world instance). In the talk, I’ll review some recent work on GP, contrast it with work on model-free approaches to the computation of general plans, and draw connections to the problem of representation learning, and in particular, representation learning for planning. This is joint work with Blai Bonet.

These talks consist of 20 minutes presentations and 5 minutes for questions.

Peter Stone

Task-Motion Planning with Reinforcement Learning for Adaptable Mobile Service Robots

Task-motion planning (TMP) addresses the problem of efficiently generating executable and low-cost task plans in a discrete space such that the (initially unknown) action costs are determined by motion plans in a corresponding continuous space. A task-motion plan for a mobile service robot that behaves in a highly dynamic domain can be sensitive to domain uncertainty and changes, leading to suboptimal behaviors or execution failures. This talk introduces a novel framework, TMP-RL, which is an integration of TMP and reinforcement learning (RL), to solve the problem of robust TMP in dynamic and uncertain domains.

Sheila McIlraith

Learning and Remembering: Planning with and without instruction

Even when someone tells you what to do, it can be hard to learn how to do it, and when critical information is left out it can be that much harder. In this talk, I'll overview my group's work on using formal languages to specify reward-worthy behaviour and how to capture reward functions in an automata-like structure we call Reward Machines. I'll show how Reward Machines enable RL agents to learn more effectively, and how Reward Machines can themselves be learned, serving as memory for RL in partially observable environments.

Giuseppe De Giacomo

Reactive Synthesis + World Model = Planning

Reactive Synthesis and planning in nondeterministic domains have many similarities. However one crucial distinction is that in Planning we consider separately the world specification (the domain) from the task specification (the goal). This distinction is important since the world model seldom changes, while the task that the agent has to accomplish changes unceasingly as is the agent acts in the world. We focus on LTL on finite traces and discuss how we can generalize the world specification from nondeterministic domains to general LTL formulas over finite and infinite traces, while keeping planning, i.e., synthesis, manageable. We also discuss how these more general forms of specifications naturally arise in the context of generalized planning.

George Konidaris

Learning Portable Skills and Symbols

I will describe my group's research on learning task-specific abstract representations that pair high-level skills with the symbolic representations required for planning with them. These results establish a close link between procedural and perceptual abstraction; I will discuss our results from a system which learns a sound and complete abstract representation of a robot manipulation task directly from sensorimotor data. I will then highlight the shortcomings of that work, and discuss our progress on defining and learning portable skills, and how to construct correspondingly portable symbolic representations that can be reused in new tasks.