(Th1). Safa AMEUR

• Thesis topic : Gesture Recognition With The Leap Motion For Screens Manipulation In a Surgical Operating Room.

• Thesis director : Pr. Med Salim BOUHLEL. 

• Co-supervisor : Dr. Anouar BEN KHALIFA.

• Institution : ENISo, University of Sousse.

• Defense date : March 25, 2021.

• Abstract : Human action recognition has been an intense research area for more than a decade. In particular, Hand Gesture Recognition (HGR) has become one of the most interesting means of touchless human-computer interaction thanks to the advancement of sensing technology. The recent introduction of novel acquisition devices, like the Leap Motion Controller (LMC), allows obtaining a very informative description of the hand pose and motion that can be exploited for accurate gesture recognition. In this thesis, we are interested in HGR approaches applied on time series data gathered from the LMC, in a context related to medical image manipulation. We introduce the first public dataset, called "LeapGestureDB" gathered with the LMC in the medical field. This dataset consists of 6600 samples. As a second contribution, we suggest a novel feature extraction method named Chronological Pattern Indexing (CPI) which encodes the temporal order of patterns that form the performed hand gesture. In a third contribution, we provide a dynamic hand gesture recognition approach using recurrent neural networks. First, we analyze the sequential time series data gathered from the LMC using different Long Short-Term Memory (LSTM) variants separately, in particular the unidirectional LSTM, the bidirectional LSTM and the deep LSTM networks. Then, we propound novel architecture by combining the aforementioned networks, named Hybrid Bidirectional Unidirectional LSTM (HBU-LSTM). The suggested network improves the model performance significantly by considering the spatial and temporal dependencies in the LMC sequential data.

Throughout this work, the recognition models are examined on two available benchmark datasets, named the LeapGestureDB dataset and the RIT dataset. We provide both quantitative and qualitative results. On several aspects related to HGR, this work outperforms the state-of-the-art gesture recognition methods in term of efficiency and computational complexity.

• Key words : Hand gesture recognition, Leap Motion controller, Feature extraction, Time series data, Chronological indexing, Deep learning, LSTM.

• Publications : This thesis led to the publication of the following papers : 

(C43). Safa Ameur, Anouar Ben Khalifa, Mohamed Ali Mahjoub, A Deep GRU-Autoencoder for Dimentionality Reduction in Hand Skeleton Data Sequences classification, The 1st IEEE Afro-Mediterranean Conference on Artificial Intelligence (IEEE AMCAI),  December 13-15, 2023 Hammamet, Tunisia.  

(C24). Safa Ameur, Anouar Ben Khalifa, Mohamed Salim Bouhlel, Hand-gesture-based Touchless Exploration of Medical Images with Leap Motion Controller, 17th IEEE International Multi-Conference on Systems, Signals and Devices (SSD’20) , 20-23 july 2020, pp. 6-11, Sfax-Tunisia. DOI: https://doi.org/10.1109/SSD49366.2020.9364244. 

(J10). Safa Ameur, Anouar Ben Khalifa, Med Salim Bouhlel, A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with Leap Motion, Entertainment Computing, Volume 35, August 2020, 100373, DOI: https://doi.org/10.1016/j.entcom.2020.100373. Quartile: Q2, IF= 1.455. 

(J9). Safa Ameur, Anouar Ben Khalifa, Med Salim Bouhlel, Chronological pattern indexing: An efficient feature extraction method for hand gesture recognition with Leap Motion, Journal of Visual Communication and Image Representation, Volume 70, July 2020, 102842, DOI: https://doi.org/10.1016/j.jvcir.2020.102842. Quartile: Q1, IF= 2.678.

(C16). Safa Ameur, Anouar Ben Khalifa, Mohamed Salim Bouhlel, LeapGestureDB: A Public Leap Motion Database Applied for Dynamic Hand Gesture Recognition in Surgical Procedures, In: Balas V., Jain L., Balas M., Shahbazova S. (eds) Soft Computing Applications. SOFA 2018. Advances in Intelligent Systems and Computing, vol 1222. pp. 125-138, Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-52190-5_9 (Conf.Rank C) 

(C10). Safa Ameur, Anouar Ben Khalifa, Mohamed Salim Bouhlel, A comprehensive leap motion database for hand gesture recognition, 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications, pp. 514 - 519, 2016. DOI: https://doi.org/10.1109/SETIT.2016.7939924