Research Projects

1. Eye Gaze Estimation with Limited Supervision: [PhD Thesis]

Automatic eye gaze estimation has interested researchers for a while now. The most promising eye gaze estimation techniques either require specialized hardware or use supervised machine learning for image processing solutions. The hardware-based methods require device calibration and operator assistance. The supervised-learning based methods require a bulk of labelled data for training. Labelling human gaze behaviour analysis data is a complex and time-consuming task. In this project, I will mainly exploit domain knowledge to labels the raw data in a generic way. Mainly it will be on representation learning in a self-supervised manner using deep learning techniques.  

Research Outcome:

2. Group Affect Analysis: [Masters Thesis]

Group Level Emotion:

Understanding of emotion of a group of people in an image or video is an imporant problem. For details on the Group-level emotion recognition, a talk by Prof. Goecke is available on video lectures at: http://videolectures.net/fgconference2015_goecke_people_images/ 

Group Level Cohesion:

Cohesiveness of a group is an essential indicator of emotional state, structure and success of a group of people. We study the factors that influence the perception of group level cohesion and propose methods for estimating the human-perceived cohesion on the Group Cohesiveness Scale (GCS). Based on the Group Affect database, we add GCS and propose the 'GAF-Cohesion database'.  It is interesting to note that GCS as an attribute, when jointly trained for group level emotion prediction, helps in increasing the performance for the later task. This suggests that group level emotion and GCS are correlated.

Most Influential Person:

Group affect analysis is an important cue for predicting various group traits. Generally, the estimation of the group affect, emotional responses, eye gaze and position of people in images are the important cues to identify an important person from a group of people. The main focus of this study is to explore the importance of group affect in finding the representative of a group. We call that person the" Most Influential Person"(for the first impression) or" leader" of a group. In order to identify the main visual cues for" Most Influential Person", we conducted a user survey. Based on the survey statistics, we annotate the" influential persons" in 1000 images of Group AFfect database (GAF 2.0) via LabelMe toolbox and propose the" GAF-personage database". 

Research Outcome:

3. Pain Estimation from Body Sensor:

Automatic chronic pain assessment and pain intensity estimation has been attracting growing attention due to its widespread applications. One of the prevalent issues in automatic pain analysis is inadequate balanced expert-labelled data for pain estimation. This work proposes an anomaly detection based network addressing one of the existing limitations of automatic pain assessment. The evaluation of the network is performed on pain intensity recognition and protective behaviour prediction tasks from body movements in the EmoPain Challenge dataset. The EmoPain dataset consists of body part based sensor data for both the tasks. The proposed network is a lightweight LSTM-DNN model which considers features based on sensor data as the input and predicts intensity level of pain and presence or absence of protective behaviour in chronic low back pain patients. Joint training considering body movement patterns, such as exercise type, corresponding to pain exhibition as a label improves the performance of the network. However, contrary to perception, protective behaviour rather exists sporadically alongside pain in the EmoPain dataset. This induces yet another complication in accurate prediction of protective behaviour. This problem is resolved by incorporating anomaly detection in the network. A detailed comparison of different models with varied features is outlined in the paper presenting a significant improvement with the final proposed anomaly detection based network.

Research Outcome:

4. Depression Intensity Estimation from Social Media:

Nowadays, depression has become a major reason of suicide, specially among the teenagers. Generally, clinical psychologists diagnose depressed people via face to face interviews following the clinical depression criteria. However, often patients tend to not consult doctors in their early stages of depression. These days people are increasingly using social media to express their mood.It has been recently found quite effective in detecting early symptoms of depression. In this paper, we aim to predict depressed users as well as estimate their depression intensity via leveraging social media (Twitter), in order to aid in raising an alarm. To this end, we weakly label the data in a self-supervised manner, extract a rich set of features, and train a small LSTM (Long Short-Term Memory) network to predict the depression intensities. We perform extensive experiments to establish the efficacy of our method.

Research Outcome:


*contributed equally.