DL research projects
DL research projects
Embedded Files
Can the Prismer VQA Model Enhance Pneumonia Diagnosis in Xrays through a Mixture-of-Experts Approach?
Can the Prismer VQA Model Enhance Pneumonia Diagnosis in Xrays through a Mixture-of-Experts Approach?
Ipsita Praharaj (iprahara) @andrew.cmu.edu , Dhwani Sanmukhani (dsanmukh), Himanshu Thakur (hthakur)
Ipsita Praharaj (iprahara) @andrew.cmu.edu , Dhwani Sanmukhani (dsanmukh), Himanshu Thakur (hthakur)
NeurIPS - Women in Machine Learning 2023, New Orlean (Abstract)
NeurIPS - Women in Machine Learning 2023, New Orlean (Abstract)
In this study, we present advancements in the NVIDIA VQA model, “Prismer,” which was fine-tuned on the NIH X-ray dataset, achieving a noteworthy enhancement in accuracy by 30% and an impressive increase in the F1 score from 0.3846 to 0.837 (a 110% improvement). Additionally, we explored the integration of ImageNet's CheXNet with Prismer through the utilization of QA-formatted X-rays. Surprisingly, this integration did not yield any gains in detection performance, highlighting the complexities of model interaction and the need for further investigation into the underlying factors affecting efficacy in integrated frameworks. These findings provide critical insights into enhancing medical imaging interpretation while also indicating challenges that may arise during model ensemble approaches.
Intelli-Research Assistant - Identifying Research Gaps in Multidisciplinary Knowledge Graphs + RAG
Intelli-Research Assistant - Identifying Research Gaps in Multidisciplinary Knowledge Graphs + RAG
Jiuyuan Xie, Ipsita Praharaj, Sahithya Senthilkumar
Jiuyuan Xie, Ipsita Praharaj, Sahithya Senthilkumar
Carnegie Mellon University, Heinz College Hybrid Human Artificial Intelligence 2024, Sweden (Abstract pending)
Carnegie Mellon University, Heinz College Hybrid Human Artificial Intelligence 2024, Sweden (Abstract pending)
In 2023, we developed an innovative research paper discovery system that leverages retrieval-augmented generation techniques with Neo4j graph database integration. Our approach involved fine-tuning large language models, specifically GPT-3.5 and LLaMA, to enhance the efficacy of information retrieval. We meticulously handcrafted 150 queries utilizing GPT-4, resulting in a significant quality improvement, evidenced by the correction of 64% of the generated queries through regular expressions. Furthermore, we employed prompt-tuning strategies that contributed to the generation of more coherent and contextually relevant summaries. This research highlights the potential of combining advanced AI models with graph database structures to facilitate more effective academic literature searches and knowledge extraction.
Page updated
Google Sites
Report abuse