RAG with Stack Overflow Posts for LLM Code Repair
RAG with Stack Overflow Posts for LLM Code Repair
Elijah Mansur - Ohio State University Department of Computer Science Engineering
Full stack engineer, AI Researcher, and entrepreneur investigating self supervised learning.
LinkedIn:
Johnson Chen - Class of 2026 at Rochester Adams High School
Aspiring AI Researcher and entrepreneur with a focus on technology and investment strategies.
LinkedIn and Email:
johnsonchen2026@gmail.com
Dr. Mohommad Wardat
Assistant Professor in the Department of Computer Science and Engineering at Oakland University
Email:
https://wardat.github.io/
wardat@oakland.edu
Abstract: Identifying, localizing, and resolving bugs in software engineering is challenging and costly. This research enhances Large Language Models' (LLMs) capabilities for bug localization and code repair using Retrieval Augmented Generation (RAG) based on dynamically collected Stack Overflow posts. These posts are searchable via a Question and Answer Knowledge Graph (KGQA). We evaluate our method on the HumanEvalFix benchmark for Python using relevant closed and open-source models. Our approach facilitates error resolution in Python coding problems by creating a searchable, embedded knowledge graph representation of bug and solution information from Stack Overflow, interlinking bugs, and solutions through semi-supervised graph construction methods. We use cosine similarity on embeddings based on LLM-synthesized summaries and algorithmic features describing the coding problem and potential solution to find relevant results that improve LLM in-context performance. Preliminary results indicate that our system enhances small open-source models' ability to effectively repair code, particularly where these models have less parametric knowledge about relevant coding problems and can leverage non-parametric knowledge to provide accurate, actionable fixes .