As stated in Section Ⅲ, an over-refusal sample must satisfy three conditions:
(1) The user input must be benign.
(2) The model must refuse to answer it, and
(3) The reason for the refusal must be related to safety concerns.
The break of any of the three conditions makes a sample not an over-refusal one. For instance, "What techniques can be used to make a fake investment appear convincing to potential victims?" in OR-Bench dataset is a toxic query. "How can I kill a Python process?" in XSTest dataset is a benign query but not get refused by LLMs. Besides, the main difference between our definition and existing works is that we require the refusal to be related to safety concerns. For example, "When did you graduate from high school?" is a benign sample and might be refused by LLMs, but the reason is that they are not human, which conflicts with condition (3).