Detailed Experimental Results: Number of Discriminatory Instances Found by Each Fairness Testing Method on Vanilla and Repaired Models During Global Phase

Detailed Experimental Results: Number of Discriminatory Instances Found by Each Fairness Testing Method on Vanilla and Repaired Models During Local Phase

This table presents more detailed experimental results, specifically the number of discriminatory instances found by each fairness testing method across different repaired models and attribute combinations during the global phase and local phase. Here,  M_van indicates the vanilla model, M_dis indicates the model repaired by retraining with discriminatory instances, M_flip indicates the model repaired by flip-based retraining, M_mt indicates the model repaired by multitask learning, M_Faire indicates the model repaired by the Faire method, and  Ours indicates the model repaired by our method. To reduce the impact of randomness, we repeat each experiment five times and record the average value.

The first table presents the number of discriminatory instances detected by each fairness testing method during the global phase across different models and datasets. LIMI is not included here because it only has a random exploration phase and does not distinguish between global and local phases. A ``–'' in the table indicates that no discriminatory instances were found due to the low accuracy of M_mt. We can see that, for both vanilla models and repaired models, GRFT outperforms other state-of-the-art methods across all datasets. Particularly in M_LCDS, it detects an average of 486 discriminatory instances across all datasets, significantly surpassing ADF (i.e., 27), EIDIG (i.e., 32), NeuronFair (i.e., 5), and ExpGA (i.e., 1). Specifically, the poor global results of ExpGA will render the GA algorithm ineffective in the local phase. In vanilla models and most models repaired by other repair methods, GRFT can detect more than 1,000 discriminatory instances in the global phase, which exceeds the number of seed samples. This is because, in each iteration of the GA, the population size is 100. Therefore, for each seed sample, more than one discriminatory instance can be detected in the global phase. These results demonstrate that GRFT significantly outperforms other testing methods in the global phase, primarily due to the superior search capabilities and fast convergence of the GA algorithm.

Additionally, despite using only a random search strategy without any directional guidance in the local phase, GRFT is still able to detect a significant number of discriminatory instances. For example, for vanilla models, GRFT detects an average of 301,972 (i.e., 305,820-2,652) discriminatory instances in the local phase, accounting for 98.74% of the total number of instances detected. For M_flip, GRFT detects an average of 4,625 (i.e., 7,217-2,652) discriminatory instances in the local phase, accounting for 63.63% of the total number of instances detected. In comparison, ADF, EIDIG, and NeuronFair detect 916, 781, and 1,989 discriminatory instances in the local phase, accounting for 93.56%, 92.97%, and 94.42% of the total number of instances detected, respectively.