Algorithmic Justice & AI: An Engine for Covert Racism

What happens when a man-made tool makes important decisions that are possibly rooted in racism? Recently, artificial intelligence has been the subject of both defense and criticism. Among other purposes, the presence of AI in criminal justice and predictive policing may be doing more harm than doling out justice in a fair, controlled way. 

According to an article from ProPublica, in the spring of 2014, Brisha Borden was charged with burglary and petty theft after taking a child’s bicycle. Along with the scooter stolen by her friend, the items were valued at 80 dollars. The year before, Vernon Prater had shoplifted over 86 dollars’ worth of tools from Home Depot. Prater had previously been convicted of attempted and successful armed robbery, but Borden had only a juvenile record with some minor crimes. 

However, when a computer program issued scores of reoffending, Borden received a much higher risk score than Prater. This was confusing, but something was amiss: Borden was a black woman, and Prater was a white man. Later, Prater broke into a warehouse and pilfered electronics worth thousands of dollars, serving another term. On the other hand, Borden was not convicted of any other crimes. The prediction was completely off-base.

Computer-operated methods of determining these scores, known as risk recidivism scores, are becoming increasingly common in the field of criminal justice. These programs, which use artificial intelligence–a branch of computer science dedicated to making machines intelligent–to function, make decisions based on huge datasets relevant to that specific decision. These datasets provide the “training” to keep the machines informed. 

However, it is virtually impossible to include every decision made in a program’s dataset, so there are always inaccuracies. One of those inaccuracies was found in the technology used in Borden and Prater’s analyses: the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS). This predictive policing program did not have sufficient data to base its calculations for risk scores, which led to misconstrued conclusions. 

The issue is that there cannot be any indication of bias or discrimination in criminal justice; the policy is that everyone is equal before the law. When a machine makes a choice that is not properly informed, the result can be problematic, as seen with Prater and Borden. To investigate this further, I built and evaluated an algorithm predicting risk recidivism scores, examining how accurate these AI models are based on the data available to them. 

Firstly, the dataset contained a list of offenders along with information about their crimes. One-hot encoding–a pre-processing technique for marking values–was used for the outlined column of interest: if a given reoffender recidivated in the last two years. Presence of the category was denoted with a 1, and absence with a 0. In this case, a 1 marked an instance of recidivism–the tendency to reoffend–and a 0 marked its absence, meaning a given offender did not reoffend. Using logistic regression, a probability prediction scheme where a predicted value lies within the range 0-1, the model determined a binary value for each of these offenders. 

Fairness was a key goal for this model to maintain; specifically, group fairness–everyone having the same chance regardless of a given grouping– and calibration fairness–an equal probability of recidivism– were kept in mind. Building the model began with splitting the dataset into training and testing groups. This was done to avoid overfitting, where a program reviews and familiarizes itself with too much data,  resulting in poor usage of new data. It is like using a study guide for a test: a test full of study guide questions means a student will not know how to apply their knowledge to new scenarios on another test. So, by splitting the data into training and test sets, the model does not overapply itself. Then, the training data was used to predict the test data, which was used to test the model. Separate accuracy scores for the training and testing data were printed from four classifiers. 

After using different classifiers–machine learning algorithms used to categorize data–to increase the accuracy of the model in predicting the binary values, the values still only showed precision; the testing values were all close together, but were altogether off-base from 100 percent. In terms of fairness, the Caucasian group received a score of 0.6659, while the African American group received 0.6721.

However, in terms of positive probability–the chance that an event will happen, or in this case, the chance of receiving a 1 for recidivism–the African American group received a score of 0.5242 compared to the Caucasian group’s 0.2339. This difference highlighted the substantially increased perceived likelihood of African American offenders to reoffend.

While this model would need thousands of additional entries to increase its accuracy, it is indicative of the general state of artificial intelligence in criminal justice. 

The possibility of bias in a field with no room for race-based discrimination is still common. However, when it comes to the proliferation of AI platforms, we may overestimate the present and underestimate the future. While many of these legal tools still contain noticeable biases and inaccuracies, the possibility of erasing these biases in the development of algorithms will not only streamline otherwise tedious decisions made by humans but also transform the roles of individuals and systems in the field of law.

References

Angwin, Julia, et al. “Machine Bias.” ProPublica, ProPublica, 23 May 2016, www.propublica.org/article/machine-bias-risk-assessments-in-criminal-

sentencing.