Introduction

Automated unit testing techniques can reduce manual effort to write unit test drivers/stubs and generate unit test inputs. However, automatically generated unit test drivers/stubs may raise many false alarms because they often over-approximate real context of a target function f and allow infeasible unit executions of f .

To alleviate this limitation, we have developed a concolic unit testing technique CONBRIO. CONBRIO generates a symbolic unit test driver of f using closely relevant functions to f . Also, CONBRIO filters out a false alarm by checking feasibility of a corresponding symbolic execution path with regard to f's symbolic calling contexts obtained by combining symbolic execution paths of f 's closely related predecessor functions.

In the experiments targeting the crash bugs of the 15 real-world C programs, CONBRIO shows both high bug detection ability (i.e. 91.0% of the target bugs detected) and precision (i.e. a true to false alarm ratio is 1:4.5). Furthermore, CONBRIO detects 14 new crash bugs in the latest versions of the 9 target C programs studied in other papers on crash bug detection techniques.

CONBRIO Real-world Unit Crash Bug Benchmark

We provide a benchmark on 67 real-world unit crash bugs (i.e. crash bugs whose buggy statement and crash site exist in a same function) of the7 C programs (bash-2.0, flex-2.4.3, grep-2.0, gzip-1.0.7, make-3.75, sed-1.17, and vim-5.0) in the SIR benchmark and 8 C programs (Perl-5.8.7, Bzip2-1.0.3, Gcc-3.2, Gobmk-3.3.14, Hmmer-2.0.42, Sjeng-11.2, Libquantum-0.2.4, and H264ref-9.3) in the SPEC 2006 benchmark. The crash bug data were collected and organized after examining all bug reports of the last 12 to 24 years (i.e. after examining bug fix commits reported since 1993 until 2017).

1. CONBRIO_unit_crash_bug_description.xlsx: This excel file describes crash bug information including faulty statements and corresponding patch information.

2. CONBRIO_unit_crash_bug_target_programs.zip: This archive contains target programs of CONBRIO Real-world Unit Crash Bug benchmark. Note that we exclude 8 target programs in the SPEC 2006 benchmark since SPEC 2006 is proprietary.

CONBRIO Experiment Data

We provide detailed experiment results and unit test drivers and unit test inputs that detect crash bugs of target functions.

1. CONBRIO_experiment_results.xlsx: This excel file contains detailed experiment results.

2. known_bugs_unit_test_drivers.zip: This archive contains unit test drivers and unit test inputs of the target programs that detect known crash bugs. Note that we exclude test drivers for SPEC 2006 benchmark programs since SPEC 2006 is proprietary.

• known_bugs_system_test_drivers.zip: This archive contains system test drivers and system test inputs for the target programs of the CONBRIO Real-world Unit Crash Bug benchmark. These system executions confirm that the crash-detecting unit test executions generated by CONBRIO are true alarms because these system test executions include the crash-detecting unit test execution generated by CONBRIO.

3. unknown_bugs_unit_test_drivers.zip: This archive contains unit test drivers and unit test inputs that detect 14 new crash bugs.

• unknown_bugs_system_test_drivers.zip: This archive contains system test drivers and system test inputs for the seven target programs from which CONBRIO detects 14 new crash bugs. These system executions confirm that the crash-detecting unit test executions generated by CONBRIO are true alarms because these system test executions include the crash-detecting unit test execution generated by CONBRIO.

4. CONBRIO_experiment_results_for_rebuttal.xlsx: This excel file contains supplement experiment results to answer reviewers' comments for rebuttal.

• function_selection_strategies.pdf: This documents describes function selection strategies other than the one CONBRIO uses to answer Q1 of reviewer1.

Contact

If you have any questions, please send an e-mail to conbrioicse2018@gmail.com