(Th7). Bilel TARCHOUN  

• Thesis topic : Deep Learning Vulnerabilities, Attacks and Robustness under Real-World Settings.

• Thesis director :  Pr. Mohamed Ali Mahjoub. 

• Co-supervisor : Dr. Anouar BEN KHALIFA, Dr. Ihsen ALOUANI

• Institution : ENISo, University of Sousse.

• Defense date :  May 21, 2025

• Abstract : Computer Vision is one of the main tasks enabled by the emergence of Artificial Intelligence and the widespread availability of powerful computation hardware, even in portable devices. And the uses of Computer Vision nowadays is widespread: Guiding intelligent transportation systems, automated quality control in industrial and agricultural tasks, securing homes and public spaces, and much more. To achieve high performance even in degraded conditions, current trends in computer vision are moving beyond the confines and limitations of single view approaches and are adopting multi-view architectures to handle increasingly challenging tasks sharing information across views. However, the impressive results of the Convolutional Neural Networks that power AI-based computer vision suffer from a significant drawback: These models are vulnerable to adversarial attacks. By adding specifically crafted noise to an image, a malicious actor can fool the victim model into misclassifying objects, or hide objects from detection. A more concerning form of these attacks is Adversarial Patches, where the noise is restricted to a small area of the image, and thus can be easily applied in real-world conditions. The threat of these attacks erode trust in the results of computer vision models, especially in safety critical applications. As a result, understanding these vulnerabilities and mitigating these attacks is a very active field of research. But most of the research efforts focus on single view computer vision, with limited forays into multi-view contexts. 

In this thesis, we contribute towards filling this gap in adversarial attack research: The first aspect of our contributions is studying adversarial patch threats in a real-world multiview context. We evaluate adversarial patch performance when faced with view angle variations, the main challenge faced by adversarial patches in multi-view contexts, and find that adversarial success rates decrease sharply when faced with large variations in view angles. Next, we evaluate the threat of current adversarial patch threats against state of the art multi-view object detectors. Our results show that multi-view object detection is partially robust against existing adversarial patches. But this robustness is not sufficient against adaptive attackers, as we demonstrate with our proposed adversarial patch against multi-view detectors, which has severely degraded the performance of multi-view detectors. 

The second aspect of our contributions is the mitigation of these adversarial patch attacks: We propose Jedi, an adversarial patch defense that leverages the high entropy inherent to adversarial patches to detect and remove them. By detecting concentrations of local entropy peaks in the input image and using an autoencoder to refine these peaks into a precise mask, Jedi is able to accurately locate adversarial patches and remove them using inpainting methods. In addition to achieving state-of-the-art results even against stealthy and adaptive patches, Jedi is also model-agnostic since it does not require any knowledge about the model to defend, and thus can be used in any computer vision application. To further extend the contributions of Jedi, we perform a qualitative analysis on the failure cases of Jedi and other adversarial patch defenses, and find that high entropy is present in most failure cases, even when the applied defense does not consider entropy as a factor. This qualitative analysis suggests deeper links between high entropy and adversarial patch attacks. We also exploit the findings of the qualitative analysis by proposing Adaptive Jedi, an enhancement of the original method that improves Jedi’s performance in challenging environments.

• Key words : Adversarial Patches, Adversarial Attacks, Adversarial Patch Defenses, Computer Vision, Multi-View, Convolutional Neural Networks, Deep Learning, Artificial Intelligence, AI Security, Entropy

• Publications : This thesis led to the publication of the following papers : 

(J20). Bilel Tarchoun, Ihsen Alouani, Anouar Ben Khalifa, Mohamed Ali Mahjoub, Nael Abu-Ghazaleh, An information-theoretic perspective of physical adversarial patches, Neural Networks, August 2024, 106590, DOI: https://doi.org/10.1016/j.neunet.2024.106590. Quartile: Q1, IF=6.0.

(C38). Bilel Tarchoun, Anouar Ben Khalifa, Mohamed Ali Mahjoub, Nael Abu-Ghazaleh, Ihsen Alouani, Jedi: Entropy-based Localization and Removal of Adversarial Patches, CVPR 2023 : IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, June 2023. DOI: https://doi.org/10.1109/CVPR52729.2023.00398 (Conf.Rank  A*)  

(C33). Bilel Tarchoun, Anouar Ben Khalifa, Mohamed Ali Mahjoub, Investigating the robustness of multi-view detection to current adversarial patch threats, 6th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP'2022), pp. 1-6, 2022, Hybrid Moncton (Canada)-Sfax (Tunisia). DOI: https://doi.org/10.1109/ATSIP55956.2022.9805870. 

(C28). Bilel Tarchoun, Ihsen Alouani, Anouar Ben Khalifa, Mohamed Ali Mahjoub, Adversarial Attacks in a Multi-view Setting: An Empirical Study of the Adversarial Patches Inter-view Transferability, International Conference on Cyberworlds (CW), pp. 299-302, 2021, Caen, France. DOI: https://doi.org/10.1109/CW52790.2021.00057 (Conf.Rank B) .