The COMPAS Algorithm: Unveiling Bias in Criminal Risk Assessment

In an era increasingly reliant on data-driven decision-making, algorithms are finding their way into various aspects of the criminal justice system. One prominent example is the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm, a tool used across the nation to assess a defendant's likelihood of recidivism. But how accurate and fair are these algorithms? A groundbreaking investigation by ProPublica shed light on the potential biases embedded within COMPAS, raising critical questions about the role of technology in sentencing and parole decisions.

What is the COMPAS Algorithm?

COMPAS, developed by Northpointe Inc., is a risk assessment tool designed to predict the likelihood of a defendant re-offending. Judges and parole officers use it to inform decisions about bail, sentencing, and supervision. The algorithm generates scores based on a defendant's answers to a questionnaire, predicting their risk of general and violent recidivism.

ProPublica's Investigation: Unmasking Algorithmic Bias

ProPublica, a non-profit investigative newsroom, conducted an in-depth analysis of the COMPAS algorithm, focusing on its application in Broward County, Florida. By analyzing over 10,000 criminal defendants, they sought to determine the algorithm's accuracy and whether it exhibited bias against certain racial groups.

Key Findings of the ProPublica Study:

• Disparate Impact: Black defendants were significantly more likely than white defendants to be incorrectly classified as high-risk, even if they did not re-offend. Conversely, white defendants were more likely to be incorrectly classified as low-risk, even if they did re-offend.
• Misclassification Rates: Black defendants who did not recidivate were almost twice as likely as their white counterparts to be misclassified as high-risk (45% vs. 23%).
• False Negatives: White defendants who re-offended were mislabeled as low-risk nearly twice as often as black re-offenders (48% vs. 28%).
• Controlling for Factors: Even after controlling for prior crimes, future recidivism, age, and gender, black defendants were 45% more likely to be assigned higher risk scores than white defendants.

In essence, while the algorithm predicted recidivism with similar accuracy for both black and white defendants (around 60%), the types of errors it made were significantly different, leading to concerns about fairness and equal justice.

Examining Previous Research on Recidivism Algorithms

ProPublica's investigation was not the first to examine the potential for bias in recidivism algorithms. Earlier studies have also hinted at these issues:

• Desmarais and Singh (2013): A review of 19 different risk assessment tools found that most had limited validation and that racial bias had not been adequately studied.
• Skeem and Lowenkamp (2016): A study of the Post Conviction Risk Assessment tool used in federal courts found that while black offenders had higher average scores, the researchers did not attribute the differences to bias.
• Meta-Analysis: A 2013 Canadian study analyzing the Level of Service Inventory observed consistently higher scores for ethnic minorities.

These studies, while varied in their conclusions, underscore the importance of ongoing scrutiny and validation of risk assessment tools to ensure they do not perpetuate existing inequalities within the criminal justice system.

Digging Into the Data: How ProPublica Analyzed COMPAS

ProPublica's analysis involved a meticulous process of data acquisition, cleaning, and analysis:

1. Data Acquisition: Obtaining two years' worth of COMPAS scores (2013-2014) from the Broward County Sheriff's Office in Florida, totaling 18,610 individuals.
2. Data Filtering: Focusing on pretrial assessments, which involved discarding assessments made later in the criminal justice process.
3. Criminal History Compilation: Gathering public criminal records from the Broward County Clerk's Office.
4. Data Matching: Matching criminal records to COMPAS records using first and last names and dates of birth, following the same methodology as a Broward County COMPAS validation study.

Defining Recidivism

ProPublica adopted a definition of recidivism aligned with Northpointe's, considering it a "finger-printable arrest involving a charge and a filing for any uniform crime reporting (UCR) code" after the COMPAS assessment. This excluded traffic tickets, municipal ordinance violations, and arrests for failing to appear in court. The analysis focused on new arrests within two years of the COMPAS assessment.

Statistical Analysis

ProPublica employed a range of statistical techniques to analyze the data:

• Descriptive Statistics: Examining the distribution of COMPAS scores across racial groups using histograms.
• Logistic Regression: Modeling the odds of receiving a higher COMPAS score, controlling for factors like race, age, criminal history, and gender.
• Cox Proportional Hazards Model: Assessing the overall predictive accuracy of COMPAS by comparing recidivism rates while controlling for time.
• Contingency Tables: Evaluating the distribution of false positives and false negatives across racial groups.

Implications and Future Directions

ProPublica's investigation into the COMPAS algorithm has had a profound impact on the conversation surrounding criminal justice reform and algorithmic accountability. Here are some key takeaways:

• Algorithmic Bias is Real: The study demonstrated that algorithms, even those designed with good intentions, can perpetuate and amplify existing biases.
• Transparency is Crucial: The lack of transparency surrounding the COMPAS algorithm's inner workings makes it difficult to identify and mitigate potential biases.
• Ongoing Evaluation is Necessary: Risk assessment tools should undergo continuous evaluation to ensure they are accurate, fair, and unbiased.
• Human Oversight is Essential: Algorithms should not be used as a substitute for human judgment, especially in high-stakes decisions like sentencing and parole.

As algorithms continue to permeate the criminal justice system, it is imperative that we address the challenges of algorithmic bias head-on. By promoting transparency, conducting rigorous evaluations, and maintaining human oversight, we can strive to create a more just and equitable system for all.

Want to delve deeper into how algorithms can impact criminal justice? Check out our article on [AI in Law Enforcement: Balancing Innovation and Ethical Concerns] (insert internal link).