(Th8). Nahla MAJDOUB BHIRI  

• Thesis topic : Towards Robust Hand Gesture Recognition: A Multi-Modal Framework with Leap Motion Controller

• Thesis director :  Dr. Anouar BEN KHALIFA.

• Co-supervisor : Dr. Safa Ameur , Dr. Ihsen ALMLOUANI

• Institution : ENISo, University of Sousse.

• Defense date : June 27, 2025 

• Abstract : Hand Gesture Recognition (HGR) has emerged as a pivotal technology in advancing humancomputer interaction, enabling intuitive and contactless control through various applications such as virtual reality, assistive technologies, and immersive robotics. Despite significant advancements, existing systems face persistent challenges in dataset diversity, discriminative feature representation, and robustness to real-world variability. A systematic literature review of numerous studies first identified a critical gap in the Leap Motion Controller (LMC), while widely adopted as a depth sensor, the LMC lacks large-scale, multi-modal datasets capturing complex bimanual gestures. This limitation hinders the development of generalized models capable of addressing real-world gesture variability. To address this, this thesis introduces a novel large-scale dataset comprising 30 dynamic and static gestures performed bimanually, synchronizing high-fidelity skeletal tracking data with temporal dynamics. To contribute to this purpose, this thesis introduces a novel large-scale multi-modal dataset containing 30 dynamic and static hand gestures performed with both hands using the LMC. This dataset enhances gesture recognition research by providing rich skeletal motion data for robust model training. Furthermore, we conduct a systematic literature review to analyze existing HGR approaches and identify key challenges in data acquisition, feature extraction, and model architectures. Building upon these insights, we proposed three distinct HGR methodologies. First, we put forward, a handcrafted feature-based approach utilizing Hilbert-Huang Transform (HHT) and Fisher Discriminant Analysis (FDA) to extract informative features and improve classification accuracy. Second, we propose a deep learning-based model combining Convolutional Neural Networks (CNNs) and Bidirectional Gated Recurrent Units (BiGRUs) for effective multi-modal feature fusion. Finally, we introduce a graph-based approach using Graph Neural Networks (GNNs) to represent hand gestures as structured graphs, enhancing spatial-temporal feature learning. The proposed models are extensively evaluated on the developed dataset, demonstrating significant improvements in recognition accuracy and robustness.

• Key words :  Hand gesture recognition, Real word challenges, Leap motion controller, Multi modal, Large scale dataset, Fisher Discriminant Analysis, Hilbert-Huang Transform, Convolutional Neural Networks, Bidirectional Gated Recurrent Units, Fusion, Graph neural network.

• Publications : This thesis led to the publication of the following papers : 

(J21). Nahla Majdoub Bhiri, Safa Ameur, Imen Jegham, Ihsen Alouani, Anouar Ben Khalifa, 2MLMD: Multi-modal Leap Motion Dataset for Home Automation Hand Gesture Recognition Systems,  Arabian Journal for Science and Engineering, Volume 50, pages 7511–7535, 2025.  DOI: https://doi.org/10.1007/s13369-024-09396-6. Quartile: Q1, IF=2.6.

(C53). Nahla Majdoub Bhiri, Safa Ameur, Imen Jegham, Ihsen Alouani, Anouar Ben Khalifa, A Deep CNN-BiGRU Network for Multi-stream Hand Gesture Recognition Framework, 2024 10th International Conference on Control, Decision and Information Technologies (CoDIT), Vallette, Malta, 2024, pp. 893-898, DOI: https://doi.org/10.1109/CoDIT62066.2024.10708341  

(J19). Nahla Majdoub Bhiri, Safa Ameur, Ihsen Alouani, Mohamed Ali Mahjoub, Anouar Ben Khalifa, Hand gesture recognition with focus on leap motion: An overview, real world challenges and future directions,  Expert Systems with Applications, 120125, April 2023.  DOI: https://doi.org/10.1016/j.eswa.2023.120125 . Quartile: Q1, IF=8.665. 

(C32). Nahla Majdoub Bhiri, Safa Ameur, Imen Jegham, Mohamed Ali Mahjoub, Anouar Ben Khalifa, Fisher-HHT: A Feature Extraction Approach For Hand Gesture Recognition With a Leap Motion Controller, 6th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP'2022), pp. 1-6, 2022, Hybrid Moncton (Canada)-Sfax (Tunisia). DOI: https://doi.org/10.1109/ATSIP55956.2022.9805899.