The Phantom of LiDAR-based Perception: Black-box Attack and Defenses

Summary

Perception plays a pivotal role in autonomous driving systems, which utilizes onboard sensors like cameras and LiDARs (Light Detection and Ranging) to assess surroundings. Recent studies (Adv-LiDAR attack) have demonstrated that LiDAR-based perception is vulnerable to spoofing attacks. However, existing attacks suffer from effectiveness and generality limitations. In this work, we perform the first study to explore the general vulnerability of current LiDAR-based perception architectures. We construct the first black-box spoofing attack based on our identified vulnerability, which universally achieves around 80% mean success rates on all target models. We further perform the first defense study, proposing CARLO that accurately detects LiDAR spoofing attacks and a robust model architecture, sequential view fusion (SVF). CARLO and SVF sucessfully reduce the attack success rates to 5.5% and 2.3%, respectively.

Target LiDAR-based Perception Model Designs:

LiDAR-based perception leverages 3D object detection models to understand driving environments, in which the models output 3D bounding boxes for detected objects. The state-of-the-art 3D detection models can be grouped into three classes, which are bird’s-eye view (BEV)-based, voxel-based, and point-wise designs. In this work, we target Baidu Apollo 5.0, PointPillars, and PointRCNN as representatives for above three classes. All the three models are open-sourced and have achieved state-of-the-art performance.

Identified Vulnerability & Black-box Attack:

Due to the limited sensor attack capability (200 points), it is hard to spoof a physically-valid front-near vehicle that requires ~2000 points in a 64-beam LiDAR point cloud. However, we find two situations where a valid vehicle contains a small number of points: 1) an occluded vehicle and 2) a distant vehicle, shown in the following figure, which are similar to human visual perception that occluded and distant objects occupy fewer pixels in our retinas.

Though LiDAR sensors share similarities with human visual perception, all three state-of-the-art classes of LiDAR-based perception models operate object detection tasks in the 3D Euclidean space different from 2D vision recognition pipelines. Inherited from general object detection model designs, they do not differentiate objects at various locations in the ground plane. As a result, we can replay the traces from occluded and distant vehicles that meet the sensor attack capability to launch attacks efficiently.

Large-scale evaluations show that such a vulnerability allows adversaries to universally achieve ~80% attack success rates on three target models with merely 60 spoofed points. Please refer to our paper to find more robustness analysis of the black-box attacks.

CARLO:

Our results show that a lack of awareness for occlusion and distancing patterns enables the proposed black-box attack. One intuitive and immediate mitigation is to detect such violations of physics. We present CARLO: oCclusion-Aware hieRarchy anomaLy detectiOn, which harnesses occlusion and distancing patterns as invariant physical features to accurately detect such spoofed fake vehicles.

Our key finding is that the free space (i.e., the space that LiDAR lasers can penetrate) of a valid vehicle's 3D bounding box should be limited. since a valid vehicle should be occluded by either other objects in front of it or its own facing surface. However, due to limited sensor attack capability, there will be a large portion of free space inside its bounding box. By further applying a hierarchical detection method, evaluation shows that CARLO can effectively reduce the attack success rate from ~80% to 5.5% without sacrificing detection accuracy. Please refer to our paper to find more details.

Sequential View Fusion (SVF):

We take a step further to explore the feasibility of embedding physical features into end-to-end learning that provides better robustness for autonomou driving systems. We find that, despite BEV or 3D representations, which are used by most models, the front view (FV) is a better representation for learning occlusion features by nature. However, prior works adopting FV are still vulnerable to the proposed attacks due to their model architecture designs’ fundamental limitations. To improve the design and further enforce the learning of occlusion features, we propose sequential view fusion (SVF), a general architecture for robust LiDAR-based perception.

SVF comprises of three modules, which are: 1) semantic segmentation: a semantic segmentation network that utilizes the FV representation to computes the point-wise confidence scores (i.e., the probability that one point belongs to a vehicle). 2) view fusion: the 3D representation is augmented with semantic segmentation scores. 3) 3D object detection: a LiDAR-based object detection network that takes the augmented point clouds to predict bounding boxes. Instead of leaving the models to learn the importance of different representations by themselves, we attach a semantic segmentation network to the raw FV data. By doing so, we enforce the end-to-end learning to appreciate the FV features, so that the trained model will be resilient to LiDAR spoofing attacks.

Large-scale evaluation shows that SVF can effectively reduce the attack success rate from to 2.3% (a 2.2 times improvement compared to CARLO) with a sligh drop of detection accuracy. We believe such a drop possibly comes from two-stage training.

FAQ

Is the black-box LiDAR spoofing attack specific to academic 3D object detection models?

No, Baidu Apollo is an open-source AV system that has over 100 partners and has reached a mass production agreement with multiple partners such as Volvo and Ford. PointPillars is originally published in CVPR 2019 by nuTonomy, an automobile company (launched its robo-taxi service in Singapore) and is further adopted by Autoware.AI, another open-source AV system that partners with ARM and Intel, etc. Besides, the uncovered vulnerability is a general problem of the 3D object detection architecture. Therefore, our proposed black-box attack should have ability to transfer to any models with similar high-level designs.

Did you evaluate the black-box attack in real world?

Yes and no. We did conduct the black-box attack in in-lab environments on a Velodyne VLP-16 PUCK LiDAR, and demonstrate its effectiveness using two fine-controlled traces. However, due to the limitation of our physical attack devices (i.e., the function genenrator), we can only spoof points in 10cm-level precision. Therefore, for the large-scale evaluation in our paper, we digitally simulated the attack traces within the sensor attack capability strictly, and leverage them to evaluate and analyze the attack effectiveness and robustness.

Is LiDAR spoofing attack practical in the real world?

Yes, as mentioned before, the LiDAR spoofing attack has been demonstrated by existing works and Adv-LiDAR to be a practical attack vector. In this paper, we adopt similar sensor attack capability observed in Adv-LiDAR as the main constraint to better represent real-world practicality.

Is it really practical to launch LiDAR spoofing attacks on victim AV dynamically?

We assume the attackers can leverage high-precision equipments, and place such attack devices at roadsides to shoot malicious laser pulses to AVs passing by, or launch attacks in another vehicle in front of the victim car (e.g., on the adjacent lane). Evaluation of such attacks on a real AV requires road tests, which is too expensive to perform. So we leave such tasks for future research work.

Does the identified vulnerability provide completeness?

No, although we demonstrate that our proposed black-box attack achieves high attack success rates, the identified vulnerability does not provide completeness. This means that there may exist other potential vulnerabilities hidden in the autonomous driving systems to be discovered and exploited. Future research directions may include verification of the deep learning models and comprehensive empirical studies to explore the underlying vulnerabilities.

Do the proposed countermeasures provide guarantees of robustness?

No, although both defenses can effectively defend against LiDAR spoofing attacks under the current sensor attack capability, our countermeasures may not work at some point with the increasing capability of sensor attacks. We argue that if attackers can spoof a set of points located in the distribution of physical invariants for valid vehicles (e.g., injecting around 1500 points into the point cloud), there is arguably no way to distinguish them at the model level and it is safer for AVs to engage emergency brakes in that situation.

Why should AV developers trust the defenses if they don't provide guarantees?

Our solutions can effectively defend against LiDAR spoofing attacks under a realistic sensor attack capability and can be easily embeded into current AV perception systems. As mentioned above, the LiDAR spoofing attack requires expensive equipments for controlling its precision. Therefore, injecting a large number of points into the pristine LiDAR point cloud in the real world is NOT practical in the near future.

Research Paper

[ArXiv Version] [USENIX Security'20] Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures

Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z. Morley Mao

To appear in the 29th USENIX Security Symposium, Boston, USA, Aug. 2020. (acceptance rate (winter) 13.0% = 62/477)

BibTex for citation:

@inproceedings {sun2020robustlidar,

title = {Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures},

booktitle = {29th {USENIX} Security Symposium ({USENIX} Security 20)},

year = {2020},

url = {https://www.usenix.org/conference/usenixsecurity20/presentation/sun},

publisher = {{USENIX} Association},

month = aug,

}

Team

Jiachen Sun, Ph.D candidate, EECS, University of Michigan

Yulong Cao, Ph.D candidate, EECS, University of Michigan

Qi Alfred Chen, Assistant Professor, CS, University of California, Irvine

Z. Morley Mao, Professor, EECS, University of Michigan

Acknowledgements