Fuzzer, Raw Data and Additional Analysis

Fuzzer

We need to discuss with our funding agency to provide the source code of the fuzzer and the compiler plugin to perform instrumentation.

So here we provide the fuzzer binary distribution and a sample configuration file together with an instrumented version of mjs for interested people to play around with.

The file is "test-mjs.tar.gz" (in the list below).

The fuzzer requires the libjvm.so and libhwloc.so as its dependencies.

(This requires the installation of the orace-jdk or open-jdk and also libhwloc-dev.)

It also relies on Redis (https://redis.io/) for storing runtime information.

The command to run the test is:

./cerebro-fuzz -c Config.toml -- ./mjs_main @@

Raw Data

Here we provide the the raw data collected from the working directories of each fuzzer.

The naming convention of the files (in the list below) is as follows:

overall_xxx_yyy.TAR.GZ

xxx is the program (project) name.

yyy is the property name.

For example, "edge" means the data is the number of edges covered over time for each of the 10 runs; "entry" means the data is the number of seeds in the queue of the fuzzer for each of the 10 runs.

Additionally, "crash_strategies" means the data is for comparing the effects of different strategies (basically the MOO model and the dynamic potential score).

The data are in txt formats.

Each line is the number of crashes/edges/queue entries discovered by the corresponding fuzzer after "line number" minutes.

Coverage

Here we also plotted the coverage for the tested programs with the 95% confidence interval (https://en.wikipedia.org/wiki/Confidence_interval).

(The shaded area is the 95% confidence interval.)

The coverage refers to the number of edges (basic block transitions) discovered by the fuzzer.

The figures are for nm(new), mjs, pngfix, sqlite, cxxfilt, nm(old), xed, radare2 (from the top left corner, going clockwise).

We can see from the figures that for nm(new), mjs and xed, even the lower bounds of the interval of Cerebro are still better than the other 2 fuzzers.

This means that Cerebro actually achieves a good coverage on these projects with high confidence.

In general, this shows Cerebro can help to find more crashes without sacrificing program coverage.