Deep Interactive Motion Prediction and Planning

For an extended version of the L4DC-2022 article, please visit: https://arxiv.org/pdf/2204.02392.pdf

Abstract

In most classical Autonomous Vehicle (AV) stacks, the prediction and planning layers are separated, limiting the planner to react to predictions that are not informed by the planned trajectory of the AV. This work presents a module that tightly couples these layers via a game-theoretic Model Predictive Controller (MPC) that uses a novel interactive multi-agent neural network policy as part of its predictive model. In our setting, the MPC planner considers all the surrounding agents by informing the multi-agent policy with the planned state sequence. Fundamental to the success of our method is the design of a novel multi-agent policy network that can steer a vehicle given the state of the surrounding agents and the map information. The policy network is trained implicitly with ground-truth observation data using backpropagation through time and a differentiable dynamics model to roll out the trajectory forward in time. Finally, we show that our multi-agent policy network learns to drive while interacting with the environment, and, when combined with the game-theoretic MPC planner, can successfully generate interactive behaviors.

Interactive Multi-Agent Prediction (IMAP) Policy (Good Performance Examples)

Interactive Multi-Agent Prediction (IMAP) Policy (Bad Performance Examples)

Interactive Planning for Lane Changes

Example of interactive prediction and planning during a lane change scenario, note how the affected agent is the only one slowing down, the rest of the agents (in gray) are unaffected by the planned trajectory (orange).