Robustness Analysis

Metrics

At the beginning, we define two killing score metrics to approximate the vulnerability of an input or a segment.

DNN Robustness Analysis

We first study the robustness of FNN models (i.e., LeNet-5) on MNIST inputs analysis.

Firstly, we use different operators under different mutation ratio to generate 200 mutants base on the LeNet-5 model. Then, we use this command to calculate KScore1 and generate single analysis report:

python sort_data.py -model_path mnist_lenet5.h5 -mutants_path lenet5-mutants -save_path result/

The following tables show the consolidated results, where each histogram gives the number of mutants whose score lie in the corresponding range.

RNN Robustness Analysis

We further study the robustness of RNN models (i.e., LSTM and GRU) on IMDB sentiment analysis.

First, we analyze the RNN robustness against the inputs. For each model (i.e., LSTM and GRU), we generate 200 mutated models by randomly selecting the static-level operators. Then we call the sort_data API to calculate the KScore1 of each input.

sort_data(ori_model_path, mutants_path, x, y, save_path)

Finally, we randomly select 200 test data to perform the correlation analysis. We use FGSM attack to generate adversarial examples on the original model (the attack is configured with 0.001 step size and 100 max attack epochs). We assume that if it is more easier to generate adversarial examples from an input (i.e., with less epochs), then the model is less robust at the input.

Table I shows the results on robustness analysis for RNN models at inputs. Column Random shows the results for the selected inputs. Column KS_1 shows the average killing score. and Column #E. shows the average number of epochs to generate an adversarial example by using the FGSM attack method. Column Correlation shows the results of statistical analysis between killing score and the number of epochs to perform a successful attack, which shows that our KS_1 is significantly correlated with robustness by adversarial attacks~(i.e., attack epochs indicate the robustness).

To clearly show the difference, based on the killing score, we further select the top 100 and last 100 inputs from the test data. Then we perform the FGSM attack on each input. The last two columns (i.e., Top 100 and Last 100) show the detailed results. When performing attacks on the inputs, we could find that it is much easier to generate an adversarial example for the top 100 inputs than the last 100 inputs. Specifically, the top 100 inputs need an average of 1.18 epochs while the last 100 inputs take an average of 16.38 epochs in LSTM. The results demonstrate that our mutation analysis is useful to measure the model robustness at specific inputs.

Then, we analyze the RNN robustness at segments of inputs. We randomly select 50 test inputs that are correctly handled by the original model. For each input, suppose the RNN model will process it with n iterations (i.e., n segments). For each of the segments, we mutate the state value with 100 times and get 100 different prediction results. Then we use sort_segment.py to calculate KScore2 of each segment.

python sort_segment.py -csv_path imdb_lstm_gf_data2_0.1.csv -save_path fnn_analysis/ -column 4

Finally, we perform the FGSM attack (with step size 0.01 and 100 max epochs) on each segment. We will analyze the correlation between the KScore2 and the number of epochs to perform a successful attack.

The segment-level analysis results are shown in Table II. Column KS2 shows the KScore2 of the all input segments. Column #E. shows the average epochs used to generate an adversarial example. The statistical analysis confirms the significantly negative correlation. The results indicate that if the KScore2 is small (resp. large), the model tends to be more robust (resp. vulnerable) at the segment. The results conrm the usefulness of our mutation analysis tool for measuring the RNN model robustness at the segment level. The results confirm the usefulness of our mutation analysis tool for measuring the RNN model robustness at the segment level.

We also give some examples of the segment-level robustness analysis. The follow figures depict three IMDB inputs' behavior difference at each segment before and after mutation. Note that, the more greater the difference, the less robust the segment is.