Quality of seeds on the Pareto frontier

In Cerebro, the fuzzer uses a multi-objective model for seed prioritization.

The seeds on the Pareto frontier of all the not yet fuzzed seeds in a certain cycle are prioritized.

We conducted extra experiments to evaluate the quality of the seeds on the Pareto frontier.

We allow Cerebro to log the information of all the seeds in queue after certain numbers of cycles.

We use fixed cycles here instead of fixed time because the time for the extra logging of seed information becomes significant when the queue gets large.

These experiments are conducted on pngfix (with and without dictionary-based mutation operations) and the fuzzer stops after 300 cycles.

The table above shows some properties of the seeds on the Pareto frontier and the entire queue after 300 cycles without applying dictionary-based mutation.

We can clearly observe that the overall quality of the seeds on the Pareto frontier are better than the average of the entire queue.

In addition, the number of the seeds on the Pareto frontier is 7.5% of the entire queue while they contribute 87.3% of the total edges covered.

Other than examining the quality of seeds on the Pareto frontier, we also log the change of average input potential score.

The figure above shows the change of average input potential score over cycles.

We can see that with dictionary-based mutation, the average potential score drops quickly.

This indicates that dictionary can help the fuzzer to quickly cover new functions.

Theoretically, the average potential score can drop to 0 when the fuzzer covers every function in the PUT.

However, in reality, this can seldom happen as some functions can never be covered without setting certain command line options.

For instance, if the fuzzer run nm without the "-C" option, it can never cover certain functions performing demangling.

This hinders the usage of input potential to some extent because the potential scores of certain seeds are contributed by such functions.

The current mitigation is: if the potential score of a seed does not change after 20 consecutive cycles, it is not used for adjusting the power schedule of that seed until it changes again.