Mastering End-to-end Autonomous Driving 

with Low-Cost Device: 

Diving into the Devils of Openpilot

Anonymous

 

Abstract

Equipped with a wide span of sensors, predominant autonomous driving solutions are becoming more modular-oriented for safe system design. Though these sensors have laid a solid foundation, most massive-production solutions up to date still fall into Level 2 (L2, Partial Driving Automation) phase. Among these, Comma.ai comes to our sight, claiming one $999 aftermarket device mounted with a single camera and board inside owns the ability to handle L2 scenarios. Together with open-sourced software of the entire system released by Comma.ai, the project is named Openpilot. Is it possible? If so, how is it made possible? With curiosity in mind, we deep-dive into Openpilot and conclude that its key to success is the end-to-end system design instead of a conventional modular framework. The model inside Openpilot is briefed as Supercombo, and it can predict the ego vehicle’s future trajectory and other road semantics on the fly from monocular input. Unfortunately, the training process and massive amount of data to make all these work are not available to the public. To achieve an intensive investigation, we try to reimplement the training details and test the pipeline on public benchmarks. The refactored network proposed in this work is referred as OP-Deepdive. For a fair comparison of our version to the original Supercombo, we introduce a dual-model deployment scheme to test the driving performance in the real world. Experimental results verify that a low-cost device can indeed achieve most L2 functionalities. In this report, we would like to share the audience with our latest findings, shed some light on the new perspective of end-to-end autonomous driving from an industrial product-level side, and potentially inspire the community to continue improving the performance based on the environments provided in this work.

 

Show Cases: Our OP-Deepdive Model on Comma2k19 Dataset

M5_demo_2.mp4

Following the front car to start.

M5_demo_0.mp4

Following the front car to start at night.

M5_startstop.mp4

Stop-and-go in an intersection.

Openpilot: Crusing in Real World (ACC)

ACC_01.mp4
ACC_02.mp4
ACC_03.mp4
ACC_04.mp4
ACC_05.mp4
ACC_06.mp4
ACC_07.mp4
ACC_08.mp4
ACC_09.mp4
ACC_10.mp4
ACC_11.mp4
ACC_12.mp4

Openpilot: Crusing in Real World (ALC)

ALC_01.mp4
ALC_02.mp4
ALC_03.mp4
ALC_04.mp4
ALC_05.mp4
ALC_06.mp4
ALC_07.mp4
ALC_08.mp4

Openpilot: Crusing in Real World (Stop-and-Go)

STOP_01.mp4
STOP_02.mp4
STOP_03.mp4
STOP_04.mp4

Openpilot: Failure Cases

change_lane_failure (1).mp4

change_lane_failure

large_curvature_steer.mp4

large_curvature_steer

left_turn_lost (1).mp4

left_turn_lost

right_turn_rail.mp4

right_turn_rail

night_bus_right.mp4

night_bus_right

night_corwd_person.mp4

night_crowd_person

night_crowd_bicycle.mp4

night_crowd_bicycle

night_cut-in.mp4

night_cut-in

night_cut-in_left.mp4

night_cut-in_left

no-lane-too-left (1).mp4

no-lane-too-left

obstacle_highway.mp4

obstacle_highway