In this study, we present advancements in the NVIDIA VQA model, “Prismer,” which was fine-tuned on the NIH X-ray dataset, achieving a noteworthy enhancement in accuracy by 30% and an impressive increase in the F1 score from 0.3846 to 0.837 (a 110% improvement). Additionally, we explored the integration of ImageNet's CheXNet with Prismer through the utilization of QA-formatted X-rays. Surprisingly, this integration did not yield any gains in detection performance, highlighting the complexities of model interaction and the need for further investigation into the underlying factors affecting efficacy in integrated frameworks. These findings provide critical insights into enhancing medical imaging interpretation while also indicating challenges that may arise during model ensemble approaches.
In 2023, we developed an innovative research paper discovery system that leverages retrieval-augmented generation techniques with Neo4j graph database integration. Our approach involved fine-tuning large language models, specifically GPT-3.5 and LLaMA, to enhance the efficacy of information retrieval. We meticulously handcrafted 150 queries utilizing GPT-4, resulting in a significant quality improvement, evidenced by the correction of 64% of the generated queries through regular expressions. Furthermore, we employed prompt-tuning strategies that contributed to the generation of more coherent and contextually relevant summaries. This research highlights the potential of combining advanced AI models with graph database structures to facilitate more effective academic literature searches and knowledge extraction.