Projects
Joint projects with EmoryHITI@Radiology
Find the detail of the projects here - http://hitilab.org/projects
Prediction of clinical event by analyzing longitudinal EHR data
Unstructured medical data analysis and integration of multimodal data can unlock the large amount of electronic healthcare records (EHR) for clinical event prediction (e.g. ER visits, hospitalization, short-term mortality). My research interest is multimodal clinical data integration and predictive modeling.
Design a temporal deep learning model for estimating short-term life expectancy of the patients by analyzing free-text clinical notes.
Developed a computerized technique for assessing treatment response to neoadjuvant chemotherapy by analyzing noninvasive DCE-MRI scans.
Proposed a framework of data analysis tools for the automatic computation of qualitative and quantitative parameters to support effective annotation of patient-specific follow-up data.
Developing a longitudinal machine learning approach to predict weight gain/loss in the context of insulin sensitivity and resistance by combining multiple omics data.
1. “Probabilistic Prognostic Estimates of Survival in Metastatic Cancer Patients (PPES-Met) Utilizing Free-Text Clinical Narratives.” [link]
2. "Assessing treatment response in triple-negative breast cancer from quantitative image analysis in perfusion magnetic resonance imaging." [link]
3. "Integrative Personal Omics Profiles during Periods of Weight Gain and Loss.", [link]
4. “Semantic annotation of 3D anatomical models to support diagnosis and follow-up analysis of musculoskeletal pathologies.” [link]
Semantic annotation of patient-specific 3D anatomical model.
My doctoral dissertation was focused on realizing a comprehensive integration between patient-specific 3D data and formalized bio-medical knowledge and exploiting the integration to drive computer assisted diagnosis decision. I proposed original approaches for part-based annotation of 3D patients’ data based on seamless integration between 3D shape modeling and machine learning techniques, and to support representation of this semantics in a machine-readable way.
I have implemented a prototype system for performing automatic semantics-driven annotation of 3D patient-specific models and their parts-of-relevance, characterized by anatomical, functional landmarks or pathological markers.
1.“Semantic annotation of 3D anatomical models to support diagnosis and follow-up analysis of musculoskeletal pathologies, [link]
2. “Generation of 3D Canonical Anatomical Models: An Experience on Carpal Bones”, [link].
3. “Semantics-driven Annotation of Patient-Specific 3D Data: A Step to Assist Diagnosis and Treatment of Rheumatoid Arthritis", [link].
4. “Combination of Visual and Symbolic Knowledge: A Survey in Anatomy”, Computers in Biology and Medicine, 80, pp 148–157, 2017.
Quantitative analysis of medical images to support diagnosis.
Interested in developing computational methods that can extract quantitative information from images, integrate diverse clinical and imaging data, enable discovery of image biomarkers, and improve clinical treatment decisions. I am leading several innovative medical image analysis research projects related to cancer diagnosis, e.g. prostate cancer aggressiveness detection, histopathologic subtype classification of brain tumor, prediction of semantic features of bone tumor. I am developing a novel computational framework that can automatically interpret implicit semantic content from multimodal and/or multiparametric radiology images to enable biomedical discovery and to guide physicians in personalized care. I am responsible for the overall design of the framework, and development, execution, verification, and validation of the systems.
1. “Transfer Learning on Fused Multiparametric MR Images for Classifying Histopathological Subtypes of Rhabdomyosarcoma”, [link].
2. "Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs." [link].
3. “Computerized Prediction of Radiological Observations based on Quantitative Feature Analysis: Initial Experience in Liver Lesions” [link].
4. “Computerized Multiparametric MR image Analysis for Prostate Cancer Aggressiveness-Assessment”.
Natural Language Processing on clinical notes
The lack of labeled data creates a data “bottleneck” for developing deep learning models for medical imaging. However, healthcare institutions have millions of imaging studies which are associated with unstructured free text radiology reports that describe imaging features and diagnoses, but there are no reliable methods for leveraging these reports to create structured labels for training deep learning models. Unstructured free text thwarts machine understanding, due to the ambiguity and variations in language among radiologists and healthcare organizations.
My research is focused in developing methods to extract structured annotations of medical images from radiology reports for training complex deep learning models.
Our method has outperformed many existing NLP algorithms on several radiology report annotation tasks (CT reports, mammography reports, US reports, and X-ray reports), as well as can infer targeted information from heterogeneous clinical notes (e.g., hospital notes, discharge summary, progress notes).
Publications and open-source code:
1. “Intelligent Word Embeddings of Free-Text Radiology Reports,” [link][code]
2. “Radiology Report Annotation using Intelligent Word Embeddings: Applied to Multi-institutional Chest CT Cohort,” [link][code]
3. "Weakly supervised natural language processing for assessing patient-centered outcome following prostate cancer treatment." [link][code]
4. "Comparative effectiveness of convolutional neural network (CNN) and recurrent neural network (RNN) architectures for radiology text report classification." [link][code available upon request]
5. A Scalable Machine Learning Approach for Inferring Probabilistic US-LI-RADS Categorization [link][code available upon request]
Fusion of Fully Integrated Analog Machine Learning Classifier with Electronic Medical Records
The objective of this work is to develop a fusion artificial intelligence (AI) model that combines patient electronic medical record (EMR) and physiological sensor data to accurately predict early risk of sepsis and cardiovascular event. The fusion AI model has two components - an on-chip AI model that continuously analyzes patient electrocardiogram (ECG) data and a cloud AI model that combines EMR and prediction scores from on-chip AI model to predict risk score. The on-chip AI model is designed using analog circuits for sepsis prediction with high energy efficiency for integration with resource constrained wearable device. Combination of EMR and sensor physiological data improves prediction performance compared to EMR or physiological data alone, and the late fusion model has an accuracy of 93% in predicting sepsis 4 hours before onset. The key differentiation of this work over existing sepsis prediction literature is the use of single modality patient vital (ECG) and simple demographic information, instead of comprehensive laboratory test results and multiple vital signs. Such simple configuration and high accuracy makes our solution favorable for real-time, at-home use for self-monitoring.
Publications and open-source code:
1. "Recurrent Neural Network Circuit for Automated Detection of Atrial Fibrillation from Raw ECG." In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), 2021. [link]
2. “Fully Integrated Analog Machine Learning Classifier Using Custom Activation Function for Low Resolution Image Classification,” [link][code]
3. "Digital Machine Learning Circuit for Real-Time Stress Detection from Wearable ECG Sensor." [link][code]
AI interpretability
Concerns about intellectual property, coupled with limited transparency on the data used for training and validation, and use of complex deep learning (DL) algorithms trained with millions of parameters makes medical AI a “black box”. As these “black box” algorithms are deployed into clinical practice, there is a critical need to empower a diverse group of end users with tools that they can use to explain model decisions so as to build trust and improve clinical adoption of AI. Our group is developing a model agnostic semantic captioning explainer network that will generate de-novo text explanations for predictions from radiology diagnostic models across heterogeneous datasets.