Mutation Strategies
We evaluate the generalization ability of DistXplore by adapting it to two NLP classification tasks: ie, sentiment analysis on IMDB and news classification on AG's News. We fine-tune the pre-trained model BERT on the two datasets, respectively.}
Due to the intrinsic differences between images and textual data, we develop the text specific mutation strategies. Mutation strategy designed for text data needs to adapt to the discrete nature of text and effectively influence the model's performance without altering the semantics. Following the mutation strategy generally used in NLP adversarial attacks, a text is mutated by replacing a certain number of words with the synonyms. In our approach, we search the words causing the most significant changes in the classification probability to be targeted for replacement in the mutation process. Specifically, for each word in a text, we calculate the change in the classification probability after replacing the word with '[UNK]' and greedily search for the words that cause the most significant changes. Each word selected to be mutated is then randomly replaced by a synonym in the synonym set to make the perturbation small enough, the synonym set is built using WordNet.