In summer 2022 I interned at the Human Centric Machine Learning Group at Max Planck Institute for Software Systems in Kaiserslautern, Germany. Under the guidance of Dr. Manuel Gomez-Rodriguez I worked on formulating a threshold test to determine discrimination in case of binary decisions taken by human decision makers.
Individuals have multiple (protected) traits that describe them, some of them observable by others. Decision makers often judge individuals on the basis of these observed features, as well as the biases that they may harbour in favour of/against certain traits. Usually more than on trait influences the final decision taken by the decision maker on an individual.
Thus it may be the case that a decision maker employs different metrics for measuring the worthiness of different individuals when assigning an outcome to them.
In binary (yes/no) decisions, we can consider the metric to be a numerical threshold for assigning the outcome yes to an individual.
We wish to infer the different threshold values that are employed by the decision maker for different subsets of features (or to see whether such a unique subset exists).
Two main tests for bias-detection in human decisions are benchmarking, outcome tests and threshold tests.
Benchmarking compares the rate of favourable decisions for each group (i.e. if one social group is more likely to be granted loans than another,
Outcome tests, instead of considering the rate of favourable decisions, consider the success rate of decisions across groups. That is, given the decisions taken, if the future outcome indicates
This paper uses police stop-and-search decisions as an illustrative example for their threshold test.
Given the race r and location d of a motorist,
the officer observes a signal p~Beta(phi, lambda) , which is interpreted as the (observed) probability of the individual carrying contraband
This paper uses police stop-and-search decisions as an illustrative example for their threshold test.
Given the race r and location d of a motorist,
the officer observes a signal p~Beta(phi, lambda) , which is interpreted as the (observed) probability of the individual carrying contraband