Replication Package

We provide the replication package of ArchRepair.

ArchRepair

The replication package to reproduce the experiments in this paper can be download here.

The package requires Python >= 3.9, and you can install all required package by using pip install -r requirements.txt.

RQ1

In RQ1, we use ArchRepair to repair the DNN model on clean datasets. Therefore, to replicate RQ1, you need to set training set as original clean dataset and keep other parameters as usual. For example, you can use the following command:

python archrepair.pyc \

--gpu 0 \

--dataset cifar10 \

--dataset-path ./data/CIFAR-10 \

--corruption-path ./data/CIFAR-10-C \

--training-set clean \

--model resnet18 \

--model-path pretrain_resnet18.tar \

--save-path ./save_model/repair/resnet-18 \

--epoch 500 \

--batch 128 \

--learning-rate 0.01 \

--momentum 0.9 \

--decay 0.0001 \

--repair block4

In RQ1 we use the different models and different datasets to get the result, so you need to change --model in 4 models (resnet18, resnet50, resnet101 or densenet121) and --dataset in 2 datasets (cifar10 or tiny_imagenet) respectively to get the result. In this way, you can get the repaired model on 4 types of architectures and 2 datasets like in RQ1.

Like the result in the RQ1, you can observe that the DNN models have higher accuracy after being repaired, which shows the repairing capability of ArchRepair.

RQ2

In RQ2, we use ArchRepair to repair the DNN model on 15 corruption datasets. To replicate RQ2, we need to set the parameter --training-set as one of the 15 corruption datasets (gaussian_noise, shot_noise, impulse_noise, defocus_blur, glass_blur, motion_blur, zoom_blur, snow, frost, fog, brightness, contrast, elastic_transform, pixelate, jpeg_compression), and keep other parameters as usual. For example, we can use the following command:

python archrepair.pyc \

--gpu 0 \

ataset cifar10 \

--dataset-path ./data/CIFAR-10 \

--corruption-path ./data/CIFAR-10-C \

--training-set gaussian_noise \

--model resnet18 \

--model-path pretrain_resnet18.tar \

--save-path ./save_model/repair/resnet-18 \

--epoch 500 \

--batch 128 \

--learning-rate 0.01 \

--momentum 0.9 \

--decay 0.0001 \

--repair block4

With this command you can observe that the accuracies of repaired models on 15 corruption datasets are all improved, which demonstrates ArchRepair can enhance the robustness of DNN models after repairing.

RQ4

In RQ4, we compare the effectiveness of ArchRepair on both layer-level and block-level. To replicate RQ4, we need to set repairing type as layer-level and block-level to get the repairing results on different architecture level respectively. For example, we can use the following command:

On block-level:

python archrepair.pyc \

--gpu 0 \

--dataset cifar10 \

--dataset-path ./data/CIFAR-10 \

--corruption-path ./data/CIFAR-10-C \

--training-set clean \

--model resnet18 \

--model-path pretrain_resnet18.tar \

--save-path ./save_model/repair/resnet-18 \

--epoch 500 \

--batch 128 \

--learning-rate 0.01 \

--momentum 0.9 \

--decay 0.0001 \

--repair block4 \

--repair-level block_level

This command repair the ResNet-18 model on block-level, we can change the parameter --repair-level to layer_level to repair the model on layer-level.

In this way you can get repaired models on two different architecture levels, and you can observe that the models repaired on block-level are always have the higher accuracy than the models repaired on layer-level, which shows ArchRepair can perform better on block-level than on layer-level.

Basic Usage

The basic usage is like follows:

python archrepair.pyc \

--gpu [GPU] \

--dataset [cifar10 or tiny_imagenet] \

--dataset-path [path of dataset's directory] \

--corruption-path [path of corruption dataset's directory] \

--training-set [clean or certain corruption dataset] \

--model [resnet18, resnet50, resnet101, densenet121] \

--model-path [path of pre-trained model] \

--save-path [path of directory for saving the results] \

--train-epoch [epoch for training] \

--train-batch [Batch size of training] \

--train-learning-rate [Learning rate of training] \

--train-momentum [Momentum of training] \

--train-decay [Decay of training] \

--epoch [Epoch of repairing] \

--batch [Batch size of repairing] \

--learning-rate [Learning rate of repairing] \

--momentum [Momentum of repairing] \

--decay [Decay of repairing] \

--repair [The name of vulnerable block] \

--repair-type [layer-level or block-level] \

Specifically, --gpu is to select which gpu to train or repair the DNN model. --dataset is to select which dataset to pre-train the model or repair the target. --dataset-path and --corruption-path is to set the path of dataset and its corresponding corruption dataset. --training-set is to select which dataset should be set as training set, you you select the original clean dataset or one of the 15 corruption dataset as training set to train or repair the DNN model. --model is to select the architecture of model and --model-path is to set the path of targeted model. --save-path is to set the directory to store the result. epoch, batch, learning rate, momentum and decay are the hyper-parameters of the pre-training and repairing. --repair is to set the vulnerable block which need to be repaired, and --repair-type is to set the repairing level, you you use archRepair to repair on layer-level or block level. You You find more details by using command python archrepair.pyc -h.