Projects
List of publications at Google Scholar profile.
Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation [paper]
[February 2022 - July 2022] - Work done at DFKI Kaiserslautern [Accepted at WACV 2023]
In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. Although recent works focused on these issues, existing classifier initialization methods do not address the background shift problem and assign the same initialization weights to both background and new foreground class classifiers. We propose to address the background shift with a novel classifier initialization method which employs gradient-based attribution to identify the most relevant weights for new classes from the classifier’s weights for the previous background and transfer these weights to the new classifier. This warm-start weight initialization provides a general solution applicable to several CISS methods. Furthermore, it accelerates learning of new classes while mitigating forgetting. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.
Cross-task Discrimination for Class-Incremental Learning
[June 2021 - December 2021] with Dr. Joost van de Weijer at CVC Barcelona
Performing experiments to study the learning of cross-task features in class incremental learning using synthetic datasets as well as real image datasets with variations in task splits.
[January 2022 - October 2022] with Dr. Hari Om Aggrawal and Dr. Vinti Agarwal at BITS Pilani [Accepted at IEEE ISBI 2023]
The size of an individual cell type, such as a red blood cell, does not vary much among humans. We use this knowledge as a prior for classifying and detecting cells in images with only a few ground truth bounding box annotations, while most of the cells are annotated with points. This setting leads to weakly semi-supervised learning. We propose replacing points with either stochastic (ST) boxes or bounding box predictions during the training process. The proposed “mean-IOU” ST box maximizes the overlap with all the boxes belonging to the sample space with a class-specific approximated prior probability distribution of bounding boxes. Our method trains with both box- and point-labelled images in conjunction, unlike the existing methods, which train first with box- and then point-labelled images. In the most challenging setting, when only 5% images are box-labelled, quantitative experiments on a urine dataset show that our onestage method outperforms two-stage methods by 5.56 mAP. Furthermore, we suggest an approach that partially answers “how many box-labelled annotations are necessary?” before training a machine learning model.
UMID - Urine Microscopic Image Dataset
[July 2020 - December 2021] with Dr. Hari Om Aggrawal and Dr. Vinti Agarwal at BITS Pilani
Image Processing Algorithms for detecting and classifying Pus, RBC and epithelial Cells from microscopic images of urine sediment samples. Implementation of YOLO‑v3 and RetinaNet architectures for detection, classification and localization of cells.
Cell detection and localization from weakly labeled annotations of cells and their clusters.
Prepared a dataset of around 366 annotated microscopic images of Urine Sediment particles.
Graph-based Approach for Enumerating Floorplans Based on Users Specifications
[August 2020 - April 2021] with Dr. Krishnendra Shekhawat at BITS Pilani
This work aims at automatically generating dimensioned floorplans while considering constraints given by the users in the form of adjacency as well as connectivity graphs. My contribution to this work was focused on generating spanning circulations within the generated floorplans to connect all the rooms by a single corridor with a user-specified entry point.
Paper accepted at AIEDAM Journal (Cambridge University Press)
Breakfast Video Action Classification
Video action classification on the Breakfast actions dataset containing 1712 videos.
The sub-action segments and labels are generated from ground truth data and used to train the Bidirectional LSTM layers with Max-pooling architecture and Softmax classification layer which performs sub-action classification. [Github]
Distribution Propagation Graph Network for few-shot Learning
[Sept. 2020 - Nov. 2020] with Dr. Kamlesh Tiwari at BITS Pilani for Machine Learning Course Term Project
Study and Implementation of the DPGN paper (CVPR 2020).
Research, Innovation and Modified Version of DPGN. Introduced dense connections between the generations of DPGN.
Monocular Depth Estimation
[June 2020 - July 2020] with Chloropy Technologies
Study and Implementation of Self-Supervised Monocular Depth Estimation to estimate plant heights from videos taken using drones and generate 3D visualizations from depth maps.
Deep Learning based Agronomic Counting
[May 2020 - June 2020] with Chloropy Technologies
Implemented YOLOv3 Deep Learning Model for Agronomic counting using drone images of crops and plant-level data.
Analysis of Word-level Embeddings for Indic Languages on AI4Bharat-IndicNLP Corpora
[July 2020 - September 2020] with Prof. Uma Shanker Tiwary at IIIT Allahabad
This paper presents the analysis of non-contextual word embeddings trained on AI4Bharat-IndicNLP corpus containing 2.7 billion words covering 10 Indian languages. We share the pre-trained embeddings for research and development in Indic languages. These embeddings are evaluated on several evaluation tasks like word similarity and analogy evaluation, classification tasks on multiple datasets. The analysis of word embeddings is expected to give researchers a better understanding of the Indic Languages. We show that Word2Vec skip-gram and fastText skip-gram embeddings are the best performing models for NLP tasks on Indic languages. All the embeddings are made freely available. [Paper]
Hindi Adaption of Eliza Chatbot
[May 2020 - July 2020] with Prof. Uma Shanker Tiwary at IIIT Allahabad
Android App on Hindi Adaption of ELIZA Chatbot (Voice-Enabled) which simulates a psychotherapist in an initial psychiatric interview.
Github: ELIZA