Selected Projects

In this work, we introduce a novel method of incorporating temporal dynamics into static image understanding. We describe a neuro-inspired method that decomposes a single image into a series of coarse-to-fine images that simulates how biological vision integrates information over time. Next, we demonstrate how our novel visual perception framework can utilize this information "over time" using a biologically plausible algorithm with recurrent units, and as a result, significantly improving its accuracy and robustness over standard CNNs. We also compare our proposed approach with state-of-the-art models and explicitly quantify our adversarial robustness properties through multiple ablation studies. Our quantitative and qualitative results convincingly demonstrate exciting and transformative improvements over the standard computer vision and deep learning architectures used today.

📋 Pre-print on arxiv.

🔧 [Python, Keras, Tensorflow, OpenPV]

In this research, we employ a graph representation model, inspired by Node2Vec, to learn and capture the continuous representation of phenotypes in the Human Phenotype Ontology (HPO) graph. This technique allows us to learn a mapping for phenotypes in a low-dimensional vector space. We augment the information from the HPO knowledge graph with data extracted from a very large patient cohort. Our preliminary results show that we provide a more accurate representation, especially for rare phenotypes, by incorporating patient data. ( This is a part of my work when I interned at CHOP.)

📋 Peresentated at MidAtlantic Bioinformatics Conference 2021.

🔧 [Python, Keras, Tensorflow, BigQuery, Numba]

In this study, we gradually give more information about the input signal, using three reconstruction methods, to a deep learning model (ResNet50) as well as human participants and compare their performance both in terms of accuracy and confidence. Our analysis shows that humans' confidence and accuracy increase as the quality of the images increases, and unlike the deep learning model, there is almost no sudden change in their accuracy or confidence.

📋 Paper at GHC 2021.

🔧 [Python, Keras, Tensorflow, OpenPV]

In this work, we investigate how a deep learning network would classify images over time. The model is gradually given more information about the input signal and must decide at each timestep. Our results show that deep learning models are very sensitive and fragile to information changes.

📋 Extended-abstract at WiCV@CVPR 2021.

🔧 [Python, Keras, Tensorflow, OpenPV]

In this work, we designed and implemented a computational model that incorporates important neural mechanisms observed in the brain, including but not limited to holistic processing and top-down feedback. We show that these characteristics provide a classification framework that is robust to out-of-distribution and bias. We demonstrate our model outperforms supervised deep neural networks in a face-detection problem.

📋 Pre-print on arxiv.

🔧 [Python, Keras, Tensorflow, OpenPV]

Here, we present an unsupervised technique that allows us to detect anomalous driving behavior in the post-concussion participants, and ultimately to provide detected anomalous behaviors and a suggested risk category as a decision input to clinicians.

📋 Paper in-preparation, Poster at CRA-WP 2021.

🔧 [Python, Keras, Tensorflow, Scikit-learn]

In this project, we improved concussed driving behavior analysis by leveraging time-related information in the driving records of individuals. Our proposed method—namely, the use of 1-D convolutional networks, for classification and analysis—could not only classify concussed driving with reliable accuracy but is also capable of extracting local features on driving sequences that translate to behavioral signatures in driver performance

📋 Paper, HFES 2020)

🔧 [Python, Keras, Tensorflow]

In this study, we showed the combination of Virtual Reality Driving Simulation (VRDS) and machine-learning classifiers such as SVM and KNN could improve accurate identification of clinically relevant driving behaviors in individuals with a concussion

📋 Extended abstract, INS 2020)

🔧 [Python, Scikit-learn]

This project was the final project for my Computer Vision course in which I implemented an automatic method for coloring gray-scale images using deep encoder-decoders and transfer learning.

📋 Report

🔧 [Python, Keras]

For my UI/UX course, I developed an online platform that could suggest a recipe based on 5 predefined mood features and also allows users to search through all recipes by an advanced search option.

💡Code ▶️ Video

🔧 [html, CSS, bootstrap, JavaScrip, PostgreSQL, Pycharm]

For my master thesis, I developed a deep spiking neural network for object recognition tasks that works in a hierarchical fashion, similar to the human visual pathway. This model is made up of two kernels on top of each other, each having a simple and a complex layer. The learning is fully unsupervised and the model extracts features over time based on the STDP learning rule. Sample features learned over time are presented in the clip.

🔧 [C++, CARLsim, CUDA]