What models and factors can lead to AI bias in the prison system?

We often hear the term "bias in AI" in the news regarding data and algorithm analysis and when discussing the newest AI innovations.  Bias is a common issue in the field of Machine Learning (ML) algorithms and can originate from many sources. To preemptively combat this, researchers and analysts always have to ask questions such as: "Is the source of the bias the AI itself?" "Is it with how we train the AI?" "Is it from larger, systemic issues?" and many others to effectively narrow down where bias comes from. Even inadvertently, marginally biased AI can have disastrous impacts on the lives of people everywhere, especially in sensitive areas such as AI-assisted surgery and criminal justice. In this project, we set out to uncover the causes of "bias in AI' through the analysis of the COMPAS dataset and various models in the sci-kit-learn library. 

In order to investigate the ins and outs of the prison system, we used the COMPAS data set, which stands for Correctional Offender Management Profiling for Alternative Sanctions. The COMPAS tool itself is a risk-assessment algorithm that determines how likely a current prisoner is to re-offend (recidivate). Created in 1998 by the private form Northpointe, this model was intended to be used in the pre-trial phase, when bail decisions are made. Ideally, based on the various characteristics of a defendant, the model would predict whether or not someone is likely to commit another crime if released. This software has been used in many U.S. jurisdictions, including New York, Wisconsin, California, and Broward County, FL. 

Bias is by no means a new topic, experienced by various groups for centuries. In the modern era, biases (and specifically racial bias) are a hot topic in the realm of U.S. criminal justice. The COMPAS project sought to replace the "human bias" of a traditional judge with an algorithm, hopefully leading to a more objective ruling. However, COMPAS has inspired significant controversy from both legal and ethical standpoints, most famously in the Wisconsin State Supreme Court case Loomis v. Wisconsin (2012). Unfortunately, COMPAS was run by a private company, and they refused to release their algorithm to the public.

In our project, we aim to use the data from the Broward County, FL public records - which ProPublica was able to obtain and make public through a Freedom of Information Act (FOIA) request - to evaluate the potential factors and models contributing to the bias observed in the system.

At a high level, we experimented with different AI models, namely Gaussian Naive Bayes, Multi-Layer Perceptron, Nearest Centroid Classifier, Ridge Regression Classifier, and Quadratic Discriminant Analysis. For each of these models, we calculated the accuracy of the recidivation decisions made by the AI based on four factors: race (African-American, Caucasian, Asian, Native American, or other), gender (Male or Female), and age category (less than 25 years old, between 25 and 45 years old, and 45 years old or above). This method, which is called the Equal Accuracy model of fairness, allowed us to view how much of an impact each characteristic had on the final decision. Each model had strengths and weaknesses, but it was possible to determine if some performed better overall than others. 

In dealing with our data, we did a consistent split of our data into training and testing data: 70-30. We did this in order to ensure that each of these models would be able to work with the same data in the same form so that we could make proper conclusions about the bias in the AI.

The following tables present the average accuracy across five machine learning models (Gaussian NB, MLP, Nearest Centroid, Ridge Regression, Quadratic Discriminant) for various subgroups within the COMPAS recidivism dataset. Each table focuses on a single factor (gender, race, or age category) and shows the average prediction accuracy for each model within that subgroup. Higher average accuracy indicates a better ability of the model to predict recidivism for that particular subgroup. Discrepancies in accuracy between subgroups within a table could suggest potential bias in the model's predictions.

Key Points

• While the average accuracy differences between racial groups in our data appear small (around 0.5%), even these seemingly minor discrepancies can have significant consequences in the real world of criminal justice. In a system where a decision on incarceration or release might hinge on an algorithm's prediction, such biases could disproportionately impact certain groups.

• It's important to note that the overall average accuracy across all models and groups falls within the range of 64% to 68%. These percentages are concerningly low, raising questions about the reliability of using such models for such high-stakes decisions. An algorithm with a 65% accuracy might incorrectly classify someone as high-risk 35% of the time, potentially leading to unnecessary incarceration.

Identifying the Best Model and Most Impactful Characteristics

• Based on the average accuracy across all groups, Multi-Layer Perceptron (MLP) emerged as the model with the highest overall performance (average accuracy around 69%). However, it's important to remember that even this model's accuracy is far from ideal.

• Our analysis suggests that age category might be the most impactful characteristic on model accuracy. The models performed best for individuals between 25 and 45 years old (around 65.88% accuracy on average). This could be due to factors not included in the data, such as criminal history or life circumstances.

Process and Learning

Through this research project, we gained valuable knowledge about the issue of bias in AI systems and the importance of data selection and model evaluation. This project also highlighted the value of open data and public scrutiny for ensuring responsible AI development.

Unanswered Questions

• Our model used a limited number of characteristics and model types. Further research could involve incorporating additional data points and more complex algorithms.

• We used the same parameters and splitting for our data, but changing these could potentially uncover bias unseen in this exploration. This would help answer the question about what the source of the bias is.

• It could be useful to dive deeper into the societal and historical factors that contribute to bias in criminal justice data, in order to approach the issue of bias in AI from a human-focused standpoint.

Future

• The insights gained from projects like this can be applied to develop better AI models geared toward the criminal justice system. By understanding how different factors influence model outputs, we can create more equitable algorithms that don't continue existing biases. 

• Similar research and findings can also be used to advocate for increased transparency in AI development, particularly in high-stakes domains like criminal justice.