We chose four state-of-the-art adversarial attack methods, i.e., FGSM (Fast Gradient Sign Method), BIM (Basic Iterative Method), Deepfool and C&W attacks, to generate adversarial examples. We used the existing Python toolkit foolbox (Foolbox: A Python toolbox to benchmark the robustness of machine learning models) to perform these attacks.

Each attack is configured with the default setting as follows:

FGSM: 1000 number of steps towards the direction of the sign of the gradient, with step size between 0 and maximum-step-size 1.

BIM: limit on the perturbation size as 0.3; step size for gradient descent as 0.05; number of iterations for each gradient descent run as 10.

Deepfool: maximum number of steps to perform as 100; limit on the number of the most likely classes considered as 10.

C&W: maximum iterations as 1000; confidence of adversarial examples as 0; learning rate for the attack algorithm as 5e-3; initial tradeoff-constant to use to tune the relative importance of distance and confidence as 1e-2.

Adversarial example detection technique

1. binary activation classifier : trained with 10,000 benign data sampled from training data of each dataset and 10,000 adversarial data generated through four attacking methods; we use the activation value of the logit layer as the input representation for binary classification.
2. mutation-based detection: the mutation rate we use for the MNIST model is 0.01, 0.001 for the CIFAR10 models, and 1e-5 for the ImageNet model; 500 high-quality (we only keep the mutation models whose accuracy on the test dataset is not lower than 90% of the accuracy of its seed model) mutated models are generated and used to calculate the label change ratio (LCR) of one specific input. We select 0.3 as the threshold of LCR for MNIST and CIFAR10 models, 0.1 as the threshold for ImageNet model, based on the detection results of a series of comparative experiments with different threshold configurations.

Adversarial example defense technique

1. defensive distillation: we first train a DNN model with the same model architecture as our original model, but with softmax output layer set at temperature 20, which form a new training dataset: original training data attached with soft labels (the probability distribution). The new training dataset is then used to train another DNN model, the distilled model, with the temperature of the softmax output layer set at 20. More details of this technique can be found in Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks.
2. label smoothing: modify the label of the training data according to the following procedures: replace the value 1 for the ground truth label with a value of 0.9, and for the other classes, replace the value of 0 with 1/10k for a k classification problem; train a DNN model with the same model architecture using original data with smoothed labels following the method in Perturbation, Optimization and Statistics.
3. feature squeezing: obtain a squeezed input from the original input image using the squeezing methods- pixel color depth reduction and spatial smoothing provided in Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks; then get the model's prediction on the squeezed input for comparison with the prediction of the original input and the ground truth label of the input.
4. input transformation: apply image transformations to the images before feeding them to the models following the methods provided in Countering Adversarial Images using Input Transformations; then compare the model's prediction of the transformed input with the ground truth label of the original input to evaluate the defense performance.
5. pixel deflection: corrupt the original input images by redistributing pixel values, meanwhile ensuring that the deflected pixels lie inside the images following the methods provided in Deflecting Adversarial Attacks with Pixel Deflection to recover the true class of input images; compare the model's prediction of the deflected images with the ground truth label of the original image to evaluate the defense performance.

3. Evaluation result summary

In the defense performance evaluation, 9,000 benign data from test data set and 9,000 adversarial data generated through attacking methods as well as benign/adversarial data generated through KuK are evaluated.

We first provide the overall success rates towards each defense technique for existing data (column Comm) and data generated by KuK (column Unco), which are also shown in Section 5.2 of the paper.

Next we provide more detailed evaluation results about the success rates of defense w.r.t each specific uncertainty type towards data generated through KuK. The row #data indicates the number of generated data for each type which satisfy the objectives (constraints) we set in Section 5.1 of the paper. The success rates of each techinique for benign/adversarial data with different types are shown separately.

We could find that different kinds of defense techniques show uneven defense capacity for data with different types. For example, binary classifier perform well on LL AEs in NIN, ResNet-20 and LeNet5 models. However, the success rates for other data types are pretty low, especially for NIN and ResNet-20. In ResNet-20 model, the success rates of data with other types are all below 0.276, where the success rates for HL AEs and HH/LH/LL BEs are less than 0.065. In NIN model, the success rates on HL AEs and LH/LL BEs are all below 0.17 and the success rates of HH AEs and BEs are about o.5. Meanwhile, defensive distillation and feature squeezing present the worst defense performance towards LL AEs in both NIN and ResNet-20 models. In LeNet5 model, the success rates for HL AEs and LH/LL BEs are all less than 0.012.

From the experiment results, we could see that data with certain types are more effective in bypassing defense techniques than other types. For example, HH and HL AEs for MobileNet model are more effective than data with other types in bypassing the defense techniques. The success rates towards HH and HL AEs are less than 0.2 for all defense techniques. LL AEs show a fairly good performance with the success rates of all defense less than 0.4. Meanwhile, HH BEs perform poorly on bypassing the defense techniques, where the minimum success rate is 0.929.