Generalization in Planning

Sylvie Thiébaux, The Australian National University (Australia)

Deep Learning for Generalised Planning

Deep learning has become the approach of choice for perception and natural language processing tasks. However, whether and how this breakthrough can carry over to tasks that are commonly seen as requiring reasoning, such as planning, is a largely open question. In this talk, I will introduce neural network architectures that exploit the structure of planning problems, as captured by traditional planning representations, to learn generalised policies and heuristic estimators using little training time and data. Despite some important limitations, these architectures demonstrate good generalisation and performance on some of the planning competition benchmark domains. I will conclude by briefly discussing how this work fits in my future agenda of building a new generation of planning and scheduling systems that combine reasoning and learning, and are scalable, robust, safe, and trusted.

Highlights Talks

These talks feature highlights from recent research on topics with strong but less explored connections with Generalization in Planning.

Luc De Raedt, KU Leuven (Belgium)

From Probabilistic Logics to Neuro-Symbolic Artificial Intelligence

A central challenge to contemporary AI is to integrate learning and reasoning. The integration of learning and reasoning has been studied for decades already in the fields of statistical relational artificial intelligence and probabilistic programming. StarAI has focussed on unifying logic and probability, the two key frameworks for reasoning, and has developed machine learning techniques to probabilistic logics.

I will argue that StarAI and Probabilistic Logics form an ideal basis for developing neuro-symbolic artificial intelligence techniques. Thus neuro-symbolic computation = StarAI + Neural Networks.

Many parallels will be drawn between these two fields and will be illustrated using the Deep Probabilistic Logic Programming language DeepProbLog.

Georgios Fainekos, Arizona State University (USA)

A Lagrange Multipliers Approach for the synthesis of Feedforward Neural Network Controllers from Temporal Logic Requirements

We present a reinforcement learning approach for designing feedback neural network controllers for nonlinear systems. Given a Signal Temporal Logic (STL) specification which needs to be satisfied by the system over a set of initial conditions, the neural network parameters are tuned to maximize the satisfaction of the STL formula. The framework is based on a max-min formulation of the robustness of the STL formula. The maximization is solved through a Lagrange multipliers method, while the minimization corresponds to a falsification problem. We present our results on a vehicle and a quadrotor model and demonstrate that our approach reduces the training time more than 50 percent compared to a baseline approach.

Armando Solar-Lezama, Massachusetts Institute of Technology (USA)

Neurosymbolic program synthesis for improved generalization

In this talk, I will describe how the combination of deep learning and program synthesis can provide new capabilities for improved generalization and better interpretability for a variety of learning tasks. The talk will show how different approaches for combining these two modes of reasoning can support different applications that benefit from the strengths of both.

Aviv Tamar, Technion – Israel Institute for Technology (Israel)

Generalization in RL - A Bayesian Perspective

How can an agent learn to quickly perform well in an unknown task? This is the basic question in reinforcement learning (RL), and is critical for any effective combination of planning and learning. We study this question under the Bayesian (a.k.a. Meta RL) setting, where the essence is generalization: given a training set of tasks from some task distribution, design an agent that quickly performs well in an unseen test task. In this talk, I will describe several approaches to generalization that we have investigated. The first approach is inductive bias - value iteration networks are neural networks that include a differentiable planning module, and generalize better by learning an explicit planning computation. The second approach is learning to explore - BOReL is an offline Bayesian RL algorithm that can learn effective exploration policies from offline data. Key to making this work is solving an identifiability problem, which we characterize. Finally, I will present recent work on the theory of generalization in Bayesian RL.