LLift

LLift: Lifting Static Analysis Bug Detection with LLMs

Leverage the LLM for Program Analysis & Bug Detection

Haonan Li, Yu Hao, Yizhuo Zhai, Zhiyun Qian

UC Riverside

Abstract

Static analysis is a widely used technique in the software world in bug detection. However, creating a precise static analysis is difficult in the real world, especially since, in practice, many complex codes are difficult to analyze. Large Language Models (LLMs) offer a promising complementary, as recent advances demonstrate remarkable capabilities in comprehending code. Intuitively, LLM’s understanding of complex code can be leveraged to make complicated code snippets become analyzable automatically. In this paper, we describe how to build a practical framework combining the LLM and static analysis, using use-before-initialization (UBI) bugs as a case study. We develop LLift, a fully automated framework that combines both a static analysis tool and an LLM. By carefully designing the procedure and prompts, we are able to overcome a number of challenges, including bug-specific modeling, the large codebase, the non-deterministic nature of LLMs, etc. Tested in a real-world scenario analyzing nearly a thousand potential UBI bugs produced by static analysis, LLift demonstrates potent capability, showcasing a reasonable precision (50%) in previously undecidable code snippets and does not pose any missed bugs. It even identified 13 new UBI bugs in the Linux kernel. This research paves the way for new opportunities and methodologies in the use of LLMs for static analysis.

Source Code: https://github.com/seclab-ucr/LLift
Paper: https://haonan.me/static/papers/LLift_oopsla.pdf

LLift: Lifting Static Analysis Bug Detection with LLMs

Leverage the LLM for Program Analysis & Bug Detection

Abstract

Demo

Prompt

Case Studies

Eval on Other LLMs

@article{li2024enhancing,

title={Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach},

author={Li, Haonan and Hao, Yu and Zhai, Yizhuo and Qian, Zhiyun},

journal={Proceedings of the ACM on Programming Languages (PACMPL), Volume 8, Issue OOPSLA1},

volume={8},

number={OOPSLA1},

year={2024},

doi={10.1145/3649828}

}