Deep Imitative Models for Flexible Inference, Planning, and Control

Nicholas Rhinehart, Rowan McAllister, Sergey Levine

ICLR 2020

Paper PDF: https://openreview.net/pdf?id=Skl4mRNYDr

Abstract:

Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficult to specify. In this paper, we propose "Imitative Models'' to combine the benefits of IL and goal-directed planning. Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals. We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals. We show that our method can use these objectives to successfully direct behavior. Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection. We also show our approach is robust to poorly-specified goals, such as goals on the wrong side of the road.

CODE And Trained Model:

https://github.com/nrhine1/deep_imitative_models

Navigated Driving With Imitative Models In CARLA

The model produces expert-like plans given candidate waypoints

Navigating With waypoints On wrong side THe of road

The model produces expert-like plans that both follow the candidate waypoints and stays on the correct side of the road.

Navigating With DEcoy waypoints

The model produces expert-like plans given candidate waypoints. The model is rarely distracted by non-expert goals.

Navigating with potholes

Setting: Imitative Planning to goals >= 25m separated by 25m, with pothole costs (Town02). Our model was trained on Town01, with no examples of potholes. At test time, we show that we can control to goals in an unseen environment without crashing, while incorporating a new cost that simulates randomized potholes to avoid.