Research Overview

Present Research Topics

1. Gait Recognition

2. Person Re-identification and Recognition

3. Image and Video De-fencing

4. Face De-identification

5. Insider Threat Detection

6. Style Transfer and Image to Image Translation using Deep Learning

7. Content-based Image Retrieval

PhD Work

I completed PhD from IIT Kharagpur under the supervision of Prof. Shamik Sural from 29th December 2011 to 6th November 2015. My research topic was Frontal Gait Recognition using RGB-D Cameras. We also collaborated frequently with Prof. Jayanta Mukhopadhyay, IIT Kharagpur, and tried to include his suggestions in our work.

Gait refers to the human pattern of walking and is considered to be a potential biometric feature for identification. Several approaches for fronto-parallel gait recognition using RGB cameras had been proposed previously. However, significant attention was not given to frontal gait recognition. A possible reason is that, shape variation of the silhouette of a walking subject cannot be effectively obtained from the frontal view RGB sequences. Research on frontal gait recognition was significantly benefited after depth cameras like Kinect were made commercially available. The reason is that, the depth data obtained from these cameras can indicate relative orientation among the various body parts. Recently, quite a few approaches have been proposed that use Kinect for extracting useful gait features. These methods, however, seem to be effective only in highly constrained application domains, where at least one complete un-occluded gait cycle is needed for recognizing each subject. But possibility of occlusion or incomplete gait cycle information cannot be ruled out in most real-world scenarios. My thesis attempts to address these challenging issues using Kinect as the surveillance camera.

As an initial step, we explored the applicability of Kinect RGB-D streams for recognizing the gait patterns of individuals. We registered the depth and RGB frames from Kinect to obtain smooth silhouette shapes along with depth information. To capture accurate gait kinematics, a gait cycle was divided into a number of key poses. Gait features extracted from partial volumetric reconstruction of the registered frames corresponding to each key pose were concatenated to generate the final feature, termed as the Pose Depth Volume.

Next, we considered a more difficult situation in which availability of a complete gait cycle is not guaranteed. A single feature extracted from an incomplete gait cycle is usually not sufficiently discriminative. Hence, we proposed hierarchical classification using three independent features: (a) soft biometric feature, which captures body structural properties (b) skeleton kinematics feature, which captures the dynamic characteristics of motion and (c) fractional gait energy image feature, which captures appearance based information.

The two above-mentioned problem scenarios assumed only one subject to be present in the camera field of view at any given time. Gait recognition becomes more challenging if multiple persons enter the surveillance zone monitored by a Kinect one after the other, thereby causing occlusion. To handle this situation, we initially determined frame correspondences between each pair of gallery and test sequences. Next, clean portions of the occluded test sequence were compared with only the matching frames of the gallery sequence using weighted similarity matching.

Finally, we also proposed a fully automated frontal gait recognition approach in a multi-camera setup, which is capable of recognizing a subject after establishing correspondences among those captured by several Kinects. Information from these Kinects was used to examine which features of a gait cycle can be utilized from the independently captured sequences. A set of soft-biometric features computed from the skeleton stream provided by Kinect SDK was used for automatic re-identification as a person switches from one camera FOV to another.

Please refer to my PhD thesis attached with this page for further details. My thesis has been review by Prof. Mark S. Nixon, University of Southampton and Prof. K.R. Ramakrishnan, Indian Institute of Science, Bangalore.

Post-doctoral Work

My research at Nanyang Technological University (NTU), Singapore from 19th July 2016 to 18th July 2017 was focused on identifying potential insider threats in an organization from activity logs. An insider threat scenario refers to a set of malicious activities that result from the misuse of an organization's systems, networks, data, and resources, either intentionally, or unintentionally. Prevention of insider threat is difficult, since trusted partners of the organization are involved in it, who have authorized access to these confidential/sensitive resources. %Insider threat related activities occur very rarely, but complete prevention of insider threat demands monitoring of user cyber-activities on a regular basis. State-of-the-art research on insider threat detection mostly focuses on developing unsupervised behavioral anomaly detection techniques with the objective of finding out anomalousness, or abnormal changes in user behavior over time. But, an anomalous activity is not necessary malicious that can lead to an insider threat scenario.

As an improvement to the existing approaches, we propose a technique for insider threat detection from time-series classification of user activities. Initially, a set of single-day features is computed from the user activity logs. A time-series feature vector is next constructed from the statistics of each single-day feature over a period of time. The label of each time-series feature vector (whether malicious, or non-malicious) is extracted from the ground-truth. To address the imbalanced nature of the insider threat data consisting of only a small number of malicious instances, we employ a cost-sensitive data adjustment technique that under-samples the non-malicious class instances randomly. Finally, a classifier is trained with this data-adjusted time-series feature set along with the label information.

We evaluate our approach using the CMU Insider Threat Data (the only publicly available insider threat data set consisting of about 14 GB web-browsing logs, along with logon, device connection, file transfer, and email log files) and got encouraging results. As classifier we employ a two-layered Deep Autoencoder neural network, and compare its performance with other popularly used classifiers: Random Forest and Multilayer Perceptron. We observe that both Deep Autoencoder and Random Forest classifiers can classify the data adjusted time-series feature set with high precision, recall, and f-score. Although Multilayer Perceptron has a high recall, it suffers from a lower precision and f-score compared to the other two classifiers.

The work is jointly supervised by Dr. Lipo Wang and Dr. Tan Yap Peng (NTU, Singapore).