Generating broken lens camera failures as a class of physics-based adversarial examples
Generating broken lens camera failures as a class of physics-based adversarial examples
Manav Prabhakar, Jwalandhar Girnar, Arpan Kusari
University of Michigan Transportation Research Institute
University of Michigan, Ann Arbor, MI
Although there has been significant research on generating physics-based adversarial samples recently, one often overlooked category of such samples stems from physical failures within the camera itself. In this study, we develop a simulated physical process to create broken lens scenarios, exemplifying a type of physics-based adversarial sample rooted in camera failures. We employ a finite element model (FEM)-based simulation to generate surface cracks by applying a stress field defined by particles constrained in a triangular mesh. With physically-based rendering (PBR), we realistically visualize the resulting glass cracks. To understand the effect of these adversarial samples, we apply real instances of broken lenses and generated adversarial samples as image filters on two different open-source datasets, KITTI and MS-COCO. We measure the detection failure rate of various classes using two object detection neural network models, YOLOv8 and Faster R-CNN. To understand the distributional differences between the images, we compute the Kullback-Leibler (K-L) divergence across three different data distributions using various image filters: a dataset we collected through a windshield crack, KITTI, and images of cats from the Kaggle cats and dogs dataset. The K-L divergence results indicate that the different broken lens filters are not significantly different in terms of distribution. Our objective with this work is to establish a robust physics-based process for generating adversarial samples.
We show how we merge the stress propagation across different nodes and then use a minimum spanning tree while preventing back-cracking
An animation of how the crack propagates in the mesh with a physically based render of the mesh at each time-step.
We see an increased amount of false positives and false negatives across all classes in the adversarial video.