Mastering End-to-end Autonomous Driving
with Low-Cost Device:
Diving into the Devils of Openpilot
Anonymous
Abstract
Equipped with a wide span of sensors, predominant autonomous driving solutions are becoming more modular-oriented for safe system design. Though these sensors have laid a solid foundation, most massive-production solutions up to date still fall into Level 2 (L2, Partial Driving Automation) phase. Among these, Comma.ai comes to our sight, claiming one $999 aftermarket device mounted with a single camera and board inside owns the ability to handle L2 scenarios. Together with open-sourced software of the entire system released by Comma.ai, the project is named Openpilot. Is it possible? If so, how is it made possible? With curiosity in mind, we deep-dive into Openpilot and conclude that its key to success is the end-to-end system design instead of a conventional modular framework. The model inside Openpilot is briefed as Supercombo, and it can predict the ego vehicle’s future trajectory and other road semantics on the fly from monocular input. Unfortunately, the training process and massive amount of data to make all these work are not available to the public. To achieve an intensive investigation, we try to reimplement the training details and test the pipeline on public benchmarks. The refactored network proposed in this work is referred as OP-Deepdive. For a fair comparison of our version to the original Supercombo, we introduce a dual-model deployment scheme to test the driving performance in the real world. Experimental results verify that a low-cost device can indeed achieve most L2 functionalities. In this report, we would like to share the audience with our latest findings, shed some light on the new perspective of end-to-end autonomous driving from an industrial product-level side, and potentially inspire the community to continue improving the performance based on the environments provided in this work.
Show Cases: Our OP-Deepdive Model on Comma2k19 Dataset
Following the front car to start.
Following the front car to start at night.
Stop-and-go in an intersection.
Openpilot: Crusing in Real World (ACC)
Openpilot: Crusing in Real World (ALC)
Openpilot: Crusing in Real World (Stop-and-Go)
Openpilot: Failure Cases
change_lane_failure
large_curvature_steer
left_turn_lost
right_turn_rail
night_bus_right
night_crowd_person
night_crowd_bicycle
night_cut-in
night_cut-in_left
no-lane-too-left
obstacle_highway