SAST Tools Selection

Selection steps

We aimed at gathering a representative set of SAST tools since it is infeasible to give a complete set of all existing tools. Therefore, we searched some tool lists from recent scientific literature [1-6] and snowballed from them, as they also recommend further lists.

Eventually, we obtained several prominent websites giving recommendations for SAST tools, including Wikipedia [7], NIST [8, 9], OWASP [10, 11], GitHub [12, 13], and Kompar [14]. This process resulted in a very substantial set of SAST tools including 576 SAST tools. The overview of these tool lists is shown in the table on the right, and details in each list are displayed in the following sheets.

sast_tools_collection_576.xlsx

After removing duplicates from these 576 tools above, we further provided the full SAST tool list (192 overall and 161 for Java) which contains the selecting steps (6 selecting criteria) mentioned in our paper as follows.

Java SAST Tools Selection

Selected tools

CodeQL. CodeQL (★ 5.6k) is released by GitHub for academic research, and it claims to discover vulnerabilities by using an industry-leading semantic-based SAST engine. In this study, we use CodeQL v2.10.2, released in August 2022.
Contrast CodeSec Scan (Contrast). Contrast is a free cloud-based SAST tool, claiming as the fastest and the most accurate Java SAST tool. Here, we use version 1.0.10.
Horusec. It is an open-source syntax-based SAST tool and claims to identify vulnerabilities simply and fast during the development process. In this study, we use v2.8.0, released in June 2022.
Insidersec Insider (Insider). Insider is a syntax-based tool. It claims to cover the vulnerabilities of OWASP Top 10. In this study, we use Insider v3.0.0, released in Jan 2021.
SpotBugs with FindSecurityBugs (SBwFSB). SBwFSB is a SAST tool equipped with semantic analysis. In particular, SpotBugs (★ 2.9k) claims to cover over 400 Java detection patterns. Its plugin, FindSecurityBugs (★ 2.0k), focuses on security audits, detecting 141 vulnerability types with over 823 unique API signatures. In this study, we use SpotBugs v4.7.0 and FindSecurityBugs v1.12.0 (released in May 2022).
Semgrep. Semgrep is a free SAST tool (★ 7.6k). It uses a combination of semantic-based and syntax-based analysis, claiming to scan code and find vulnerabilities efficiently. Here, we use the community version v0.108.0 of Semgrep, released in August 2022.
SonarQube. SonarQube is a free SAST tool, which claims an industry-leading vulnerability detection with a semantic-based technology. We use its community edition v9.5.0 with sonarscanner v4.1.0, released in June 2022.

References

[1] Marcus Nachtigall, Michael Schlichtig, and Eric Bodden. 2022. A Large-Scale Study of Usability Criteria Addressed by Static Analysis Tools. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual, South Korea) (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 532-543. https://doi.org/10.1145/3533767.3534374

[2] Bushra Aloraini, Meiyappan Nagappan, Daniel M. German, Shinpei Hayashi, and Yoshiki Higo. 2019. An empirical study of security warnings from static application security testing tools. Journal of Systems and Software 158 (2019), 110427. https://doi.org/10.1016/j.jss.2019.110427

[3] Katerina Goseva-Popstojanova and Andrei Perhinschi. 2015. On the capability of static code analysis to detect security vulnerabilities. Information and Software Technology 68 (2015), 18–33. https://doi.org/10.1016/j.infsof.2015.08.002
[4] Andrew Habib and Michael Pradel. 2018. How Many of All Bugs Do We Find? A Study of Static Bug Detectors. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (Montpellier, France) (ASE 2018). Association for Computing Machinery, New York, NY, USA, 317–328. https://doi.org/10.1145/3238147.3238213

[5] David A. Tomassi. 2018. Bugs in the Wild: Examining the Effectiveness of Static Analyzers at Finding Real-World Bugs. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Lake Buena Vista, FL, USA) (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA, 980–982. https://doi.org/10.1145/3236024.3275439

[6] Sindre Beba and Magnus Melseth Karlsen. 2019. Implementation analysis of open-source Static analysis tools for detecting security vulnerabilities. Master’s thesis. NTNU.

[7] Wikipedia. 2022. Wikipedia: List of tools for static code analysis. https: //en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis (Accessed on 22/08/2022).

[8] National Institute of Standards and Technology. 2022. SAMATE: Source Code Security Analyzers. https://www.nist.gov/itl/ssd/software-quality-group/ source-code-security-analyzers (Accessed on 22/08/2022)

[9] National Institute of Standards and Technology. 2022. SAMATE: Source Code Security Analyzers. https://www.nist.gov/itl/ssd/software-quality-group/ source-code-security-analyzers (Accessed on 22/08/2022)

[10] OWASP. 2022. Free for Open Source Application Security Tools. https://owasp. org/www-community/Free_for_Open_Source_Application_Security_Tools (Accessed on 22/08/2022).

[11] OWASP. 2022. Source Code Analysis Tools. https://owasp.org/wwwcommunity/Source_Code_Analysis_Tools (Accessed on 22/08/2022).

[12] GitHub. 2022. GitHub-analysis-tools-dev. https://github.com/analysis-tools-dev/static-analysis#java (Accessed on 22/08/2022).

[13] GitHub. 2022. Awesome static analysis (multiple languages). https://github.com/mre/awesome-static-analysis#multiple-languages-1 (Accessed on 22/08/2022).

[14] Kompar. 2022. Software analyzers catalogs. https://kompar.tools/ (Accessed on 22/08/2022).

Page updated

Google Sites

Report abuse