Securing the Perception of Advanced Driving Assistance Systems Against Digital Epileptic Seizures Resulting from Emergency Vehicle Lighting
Elad Feldman, Jacob Shams, Satoru Koda, Yisroel Mirsky,
Assaf Shabtai, Yuval Elovici, Ben Nassi
Ben-Gurion University of the Negev, Fujitsu Limited
Abstract
The safety of autonomous cars has come under scrutiny in recent years, especially after 16 documented incidents involving Teslas (with autopilot engaged) crashing into parked emergency vehicles (police cars, ambulances, and firetrucks). Despite widespread public interest, the technical factors behind these accidents have remained largely unexplored. In this research, we unveil EpileptiCar, a digital epileptic seizure phenomenon that causes an object detector's confidence score to fluctuate when exposed to an activated emergency vehicle flasher. This vulnerability poses a significant risk, because not only does it cause autonomous vehicles to crash near emergency vehicles, but it can also be exploited by adversaries to cause such accidents. We delve into the underlying causes of the EpileptiCar phenomenon, assessing five commercial advanced driving assistance systems (HP, Pelsee, AZDOME, Imagebon, Rexing), four object detectors (YOLO, SSD, RetinaNet, Faster R-CNN), and 14 patterns of emergency lights to understand the influence of different factors. To mitigate this risk, we propose Caracetamol, a robust framework designed to enhance the resilience of object detectors against the effects of activated emergency vehicle flashers. Our evaluation shows that the implementation of Caracetamol improves the detection capabilities of three object detectors (YOLO, Faster R-CNN, and SSD) by 0.21 and reduces the confidence fluctuation range by 0.19, significantly enhancing safety in the presence of emergency lights.
Findings
Analysis in the Time Domain
Insight 1: The activation of emergency vehicle lighting creates a phenomenon, which we term the EpileptiCar phenomenon, that causes the confidence score of object detectors regarding a detected object (in our case it is a car) to fluctuate, within a wide score range, with the score dipping below a reasonable detection threshold in some cases.
Analysis in the RGB Space
Insight 2: The light emitted from the emergency vehicle lighting changes the distribution of the colors of the car in the captured frame, which changes the confidence of the object detector regarding the car.
The Influence of the ADAS
Samsung Galaxy
Rexing
Pelsee
Imagebon
HP
AZDome
Insight 3: The EpileptiCar phenomenon is consistent across different ADASs, with the score fluctuation ranges and stability when the flasher is off differing depending on the ADAS recording the video frames
The Influence of the Object Detector Algorithm
SSD
YOLOv9
Faster R-CNN
Insight 4: The EpileptiCar phenomenon is consistent across different object detectors, but the score fluctuation ranges differ depending on the object detector
The Influence of the Pattern of the Flasher
Insight 5: Different patterns affect object detection algorithms differently. Some patterns significantly degrade the performance of object detection.
Countermeasure (Caracetamol)
Object detection models’ receiver operating characteristic (ROC) curves for three CNN-based object detection models (mixed, Berkeley, and DashCam) when tested on the task of emergency vehicle lighting detection. The legend also includes each model’s area under the curve (AUC).
FAQ
What is an object detector?
An object detector is a computer vision model designed to identify and locate specific objects within an image or video by drawing bounding boxes around them and providing a classification of their types (e.g., a car, a road sign, a pedestrian) and associated confidence score.
A confidence score represents the model's certainty about the presence of a specific object in a detected region, usually expressed as a probability between 0 (low certainty) and 1(high certainty).
A pre-defined threshold (0-1) is used to determine whether the detected object should be considered by the system as an object or not based on the confidence score.
What is the EpileptiCar phenomenon?
The EpileptiCar phenomenon refers to the fluctuation in an object detector's confidence score when exposed to activated emergency vehicle lighting. This fluctuation can lead to significant drops in confidence (below the predefined detection threshold), potentially causing the object detector to fail to identify the emergency vehicle entirely.
How does the EpileptiCar phenomenon occur?
The EpileptiCar phenomenon occurs due to the intense, flashing light emitted from emergency vehicle lighting. This light alters the tonal distribution of the vehicle's colors in the camera frame, causing the object detector to misinterpret the visual data and resulting in fluctuations in its confidence score over time (affected by the flasher pattern).
Is the EpileptiCar phenomenon limited to specific object detectors or ADAS systems?
No, the EpileptiCar phenomenon has been observed across a variety of commercial Advanced Driver-Assistance Systems (ADAS) and popular open-source object detectors, including YOLO, SSD, RetinaNet, and Faster R-CNN.
This issue generalizes between various state-of-the-art object detectors and commercial ADAS.
What factors influence the severity of the EpileptiCar phenomenon?
Several factors can influence the severity of the EpileptiCar phenomenon, including:
Ambient light conditions: The phenomenon is more pronounced in low-light conditions, where the flashing lights have a greater impact on the camera's perception.
Emergency flasher patterns: Different flashing patterns can result in varying levels of confidence score fluctuation.
Camera settings: Shutter speed and ISO settings can impact the camera's sensitivity to the flashing lights.
Distance to the emergency vehicle: The effect is most pronounced at certain distances, where the flasher saturates the image without obscuring the vehicle entirely.
Can object trackers compensate for the EpileptiCar phenomenon?
Unfortunately, object trackers, which are often employed in ADAS to track the presence of multiple objects, have not been successful in mitigating the detection loss caused by the EpileptiCar phenomenon.
What is Caracetamol and how does it address the EpileptiCar phenomenon?
Caracetamol is the combination of the words "Car" and "Paracetamol" (a medicine used against headaches).
Caracetamol is a software framework designed to enhance the resilience of object detectors against the EpileptiCar phenomenon. It utilizes a multi-layer architecture, including:
1) a classifier to identify the presence of flashing lights
2) a layer of refined object detectors trained on augmented data containing emergency vehicles with activated flashers.
3) a layer that combines the outputs of these refined detectors with the original object detector.
Caracetamol significantly improves detection accuracy and reduces confidence score fluctuations.
Did you disclose your findings?
Yes.
We disclosed our findings with NHTSA, Tesla and the five manfuctorers of the ADAS analyzed in our study.
We hope that the publication of the study will reveal our findings to the rest of the automotive and ADAS industry.
Which industries may be affected by your findings?
Mostly the car industry.
However, any company that designs a robot whose perception is based on video may gain insights from our findings.
Could EpilptiCar be the underlying cause of the 16 documented incidents in which Teslas, with autopilot engaged, crashed into parked emergency vehicles?
We do not know.
Tesla vehicles fuse data obtained by additional sensors (e.g., RADARs and ultrasonic sensors) beyond the video cameras. Our work only focused on video cameras.
Moreover, lacking access to the specific type of object detectors and video cameras deployed in Tesla, we perform our analysis based on commonly used object detectors with other commercial ADASs available on Amazon. For this reason, there may be a discrepancy between our findings and the actual technological factors underlying the documented accidents of Tesla.
Therefore, this question should be referred to Tesla or the NHTSA.
We hope that this paper will encourage the automotive industry to validate our findings on their ADASs and semi-autonomous cars with their object detectors.
Why did the RADAR not identify the emergency vehicles?
Again, we do not know.
We lack access to the data.
This question should be referred to Tesla or the NHTSA.
Maybe the following link can give some hints.
Is EpilptiCar considered a security or safety issue?
It can be both, depending on whether the emergency flasher has intentionally (security issue) or unintentionally (safety) placed on the scene before it affected the car.
Therefore, it is crucial to handle it.
Is this the first time you have investigated the security of perception?
No. We published a study on split-second phantom attacks that demonstrated time-domain adversarial attacks against Tesla Model X a few years ago.
We demonstrated how attackers can trigger Tesla's autopilot to stop in the middle of the road in response to a stop sign that has been embedded by attackers into a digital advertisement for a few milliseconds (split second).
Press