Learning Driving Decisions by Imitating Drivers’ Control Behaviors

Junning Huang*, Sirui Xie*, Jiankai Sun, Qiurui Ma, Chunxiao Liu, Jianping Shi, Dahua Lin, Bolei Zhou

SenseTime Research University of California, Los Angeles,

The Chinese Univsersity of Hong Kong The Hong Kong University of Science and Technology


Classical autonomous driving systems are modularized as a pipeline of perception, decision, planning and control. The driving decision plays a central role in processing the observation from the perception as well as directing the execution of downstream modules. Commonly it is designed to be rule-based and difficult to learn from data. Recently end-to-end neural control policy has been proposed to replace this pipeline, given its generalization ability. However it remains challenging to enforce physically or logically constrained behaviors. In this work, we propose a hybrid framework for learning a decision module, which is agnostic to the mechanisms of perception, planning, and control modules. By imitating the low-level control behavior, it learns the high-level driving decisions while bypasses the ambiguous annotation of high-level driving decisions. We demonstrate that the simulation agents with a learned decision module can be generalized to various complex driving scenarios where the rule-based approach fails. Furthermore, it is able to generate driving behaviors that are smoother and safer than end-to-end neural policies.