Active learning is an established technique to reduce the labeling cost to build high-quality machine learning models. A core component is the acquisition function that determines which data should be selected to be labeled. State-of-the-art acquisition functions -- and more largely, active learning techniques -- have been designed to maximize clean performance (e.g. accuracy) and have disregarded robustness, an important quality property that has received increasing attention. Active learning, therefore, produces models that are accurate but not robust.

In this project, we propose robust active learning, an active learning process that integrates adversarial training -- the most established method to produce robust models.

Experimental protocol:

1. We conduct an empirical study on 11 acquisition functions, 4 datasets, 6 DNN architectures, and 15105 trained DNNs to check the performance of these functions in active learning.

2. We investigate the characteristics of selected data to explore hints of useful data for robust active learning.

3. We propose the first acquisition function for robust active learning, the density-based robust sampling with entropy (DRE), and demonstrate the efficiency of DRE.

4. We apply DRE to test selection for model retraining.