Research Publications

Predicting the Subjective Responses’ Emotion in Dialogues with Multi-Task Learning

(Oral Presentation)

Hassan Hayat, Carles Ventura and Agata Lapedriza

Anticipating the subjective emotional responses of the user is an interesting capacity for automatic dialogue systems. In this work, given a piece of a dialog, we addressed the problem of predicting the subjective emotional response of the upcoming utterances (i.e. the emotion that will be expressed by the next speaker when the speaker talks). For that, we also take into account, as input, the personality traits of the next speaker. 

Code available at: https://github.com/HassanHayat08/Predicting-the-Subjective-Responses-Emotion-in-Dialogues-with-Multi-Task-Learning

Read Paper at: https://link.springer.com/chapter/10.1007/978-3-031-36616-1_55

Modeling Subjective Annotations with  Multitask Learning

Hassan Hayat, Carles Ventura and Agata Lapedriza

In supervised learning, the generalization capabilities of trained models are based on the available annotations. Usually, multiple annotators are asked to annotate the dataset samples, and then, the common practice is to aggregate the different annotations by computing average scores or majority voting, and train and test models on these aggregated annotations. However, this practice is not suitable for all types of problems, especially when the subjective information of each annotator matters for the task modeling.  

Code available at: https://github.com/HassanHayat08/Modeling-Subjective-Affect-Annotations-with-Multi-Task-Learning

Read Paper at: https://www.mdpi.com/1424-8220/22/14/5245

Recognizing Emotions evoked by Movies using Multitask Learning

Hassan Hayat, Carles Ventura and Agata Lapedriza

In this paper, we model the emotions evoked by videos in a different manner: instead of modeling the aggregated value we jointly model the emotions experienced by each annotator and the aggregated value using a multi-task learning approach. Concretely, we propose two deep learning architectures: a Single-Task (ST) architecture and a Multi-Task (MT) architecture.  

Code available at: https://github.com/HassanHayat08/Recognizing-Emotions-evoked-by-Movies-usingMultitask-Learning

Read Paper at: https://arxiv.org/abs/2107.14529

On the Use of Interpretable CNN for Personality Trait Recognition from Audio

Hassan Hayat, Carles Ventura and Agata Lapedriza

We explored a Convolutional Neural Network (CNN) based architecture that learns the audio cues to predict the Big Five personality traits score of a speaker. In addition, we interpret our model and generate the visual correlation between the model parameters and learned representations by exploring the Class Activation Maps (CAM).  

Code available at: https://github.com/HassanHayat08/Interpretable-CNN-for-Big-Five-Personality-Traits-using-Audio-Data.git

Read Paper at: http://ebooks.iospress.nl/volumearticle/52829

Batch Regularization to Converge the Deep Neural Network for Indoor RGBD Scene Understanding

Hassan Hayat, Yazhou Liu, Maqsood Shah, Adnan Ahmad

Training a deep neural network is complicated due to the input distribution of each layer changes during training. Small changes are amplified throughout the network and consequently the covariate-shift is likely to occur. That is why small learning is critical but small learning rates is the root to slow training process and may even prevent the escape of suboptimal local minima. This paper purposed a normalization step before fusing the classification scores of different strides not the feature vectors.

Read Paper at: http://dx.doi.org/10.1109/icivc.2017.7984665.

Change Detection of Remote Sensing Image Based on Deep Neural Networks

Yan Chu, Guo Cao, Hassan Hayat

How to improve the quality of difference image (DI) for change detection task is an important issue in remote sensing images. This paper propose a new DI creation method based on deep neural networks. Deep belief network (DBN) which is an important model in deep learning is applied, and back propagation (BP) algorithm is improved according to change detection task in our method.

Read Paper at: https://www.atlantis-press.com/proceedings/aiie-16/25866363

Rotation-Based Object-Oriented Ensemble in Land Cover Classification of Hyperspactral Data

Maqsood Shah, Yazhou Liu, Hassan Hayat 

To classify hyperspectral images by using different techniques many classifiers have been produce better performance for object-oriented classification of hyperspectral remote sensing images. To improve the classification accuracy, first time we investigated an ensemble principle named RotationBased object-oriented classification of hyperspectral images (RoBOO). It is the combination of segmentation with support vector machine and nearest neighborhood algorithm. It uses random features selection and data transformation (PCA), technique to improve accuracy and diversity. 

Read Paper at: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7845045