(Th6). Karam ABDULLAH  

• Thesis topic : Driver Action Recognition At Night Time

• Thesis director :  Dr. Anouar BEN KHALIFA.

• Co-supervisor : Dr. Imen JEGHAM

• Institution : Higher Institute of Computer Science and Communication Techniques of Hammam Sousse, University of Sousse.

• Defense date :  May 27, 2023  

• Abstract : Driver action recognition is an essential task in the field of action recognition and video processing, especially with the increasing number of vehicles on the road and the need for safe transportation. The challenge is even more significant at nighttime when visibility is reduced, fatigue and sleepiness are common, and the rate of accidents increases. In this PhD thesis, four novel deep-learning approaches for driver action recognition at nighttime are proposed. These approaches are sequentially developed, building upon each other to improve accuracy. The first approach is the multi-convolutional stream for a hybrid network, which effectively fuses multimodal data to classify driver actions in low visibility and cluttered driving scenes. This approach outperforms the state-of-the-art methods by up to 10%. The second approach captures the relevant dynamic spatial information of cluttered driving scenes under low illumination, resulting in a 13% improvement in classification accuracy compared to the state-of-the-art methods on both side and front views. The third approach focuses on driver motion and uses a batch split unit to consider only relevant temporal information, resulting in a 20% improvement in recognition accuracy compared to the state-of-the-art methods. Finally, a multi-view driver action recognition system is proposed to address the challenges of single-view driver monitoring, including vision occlusion and illumination variation. The proposed approach fuses information from multiple views to provide more abundant and complementary information on driver behaviour, leading to a 28% improvement in recognition accuracy compared to the state-of-the-art methods. The experiments are conducted on the unique, public and realistic 3MDAD dataset recorded at nighttime.

• Key words :  Deep learning, Hybrid network, Driver monitoring, Nighttime, Spatio-multitemporal attention, Fusion strategies, Hard attention, Two convolutional streams.

• Publications : This thesis led to the publication of the following papers : 

(C36). Abdullah, K.; Jegham, I.; Mahjoub, M. and Anouar Ben Khalifa. Hard Spatial Attention Framework for Driver Action Recognition at Nighttime. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence (ICAART '23),  February 2023, Lisbon, Portugal, Volume 3, pages 964-971,.  DOI:  10.5220/0011846100003393 (Conf.Rank  B) 

(C35). Abdullah, K.; Jegham, I.; Mahjoub, M. and Anouar Ben Khalifa. Hard Spatio-Multi Temporal Attention Framework for Driver Monitoring at Nighttime.  In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods (ICPRAM'23), February 2023, Lisbon, Portugal, pages 51-61. DOI:  10.5220/0011637400003411  (Conf.Rank C) 

(C31). Karam ABDULLAH, imen jegham, Anouar Ben Khalifa, Mohamed Ali MAHJOUB, A Multi-Convolutional Stream for Hybrid network for Driver Action Recognition at Nighttime, 8th International Conference on Control, Decision and Information Technologies (CoDIT), pp. 337-342, 2022, Istanbul, Turkey. DOI: https://doi.org/10.1109/CoDIT55151.2022.9804013 (Conf.Rank C) .