Ground Truth Drivable Area Segmentation
Ground Truth Object Detection
Ground Truth Lane Line Segmentation
No Attack Drivable Area Result
No Attack Object Detection Prediction Result
No Attack Lane Line Segmentation Result
FGSM Epsilon 0.5 Drivable Area Result
FGSM Epsilon 0.5 Drivable Area Result
FGSM Epsilon 0.5 Lane Line Segmentation Result
Ground Truth Drivable Area Segmentation
Ground Truth Object Detection
Ground Truth Lane Line Segmentation
No Attack Drivable Area Segmentation Result
No Attack Object Detection Prediction Result
No Attack Lane Line Segmentation Result
JSMA 101376 Pixels (out of 921600) Perturbed Epsilon 0.5 DA Result
JSMA 101376 Pixels Perturbed Epsilon 0.5 Det Result
JSMA 101376 Pixels Perturbed Epsilon 0.5 LL Result
Ground Truth Drivable Area Segmentation
Ground Truth Object Detection
Ground Truth Lane Line Segmentation
No Attack Driveable Area Result
No Attack Object Detection Prediction Result
No Attack Lane Line Segmentation Result
UAP Epsilon 1.0 Step Decay 1.0 DA Result
UAP Epsilon 1.0 Step Decay 1.0 Det Result
UAP Epsilon 1.0 Step Decay 1.0 LL Result
Ground Truth Drivable Area Segmentation
Ground Truth Object Detection
Ground Truth Lane Line Segmentation
No Attack Drivable Area Segmentation Result
No Attack Object Detection Prediction Result
No Attack Lane Line Segmentation Result
CCP Color Channel Blue Epsilon 0.05 DA Result
CCP Color Channel Blue Epsilon 0.05 Det Result
CCP Color Channel Blue Epsilon 0.05 LL Result
< Here we can see some of the inputs into the GAN, and the resulting synthesized images produced by it.
The GAN is based on Pix2PixHD, created by NVIDIA. We trained our model on 500 FGSM adversarial examples (perturbed from BDD100k's training image set) taken as "labels" and 500 matching "real" images from BDD100k.
We attempted training at many different resolutions, but ultimately found that the image resolution had no true impact on YOLOP's validation.
Interestingly enough, our final results show that the perturbed images actually perform better under YOLOP's validation than the GAN synthesized version of them.
See graph here >
See table below
DEFENSE GAN Training Example
FGSM Perturbed "Label" Image
GAN Synthesized Image
"Real" Image From BDD100K