The Bias Behind the Algorithm: An In-Depth Look at COMPAS Recidivism Analysis

In an era increasingly reliant on data-driven decision-making, algorithms are playing a growing role in the criminal justice system. These algorithms are used to assess a defendant’s likelihood of re-offending, influencing decisions about bail, sentencing, and parole. However, concerns about fairness and bias in these algorithms have sparked critical debate. ProPublica, an independent, non-profit newsroom dedicated to investigative journalism, conducted an in-depth analysis of one such algorithm, COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), to evaluate its accuracy and potential bias. This article delves into ProPublica's analysis, exploring the COMPAS algorithm, its implications, and the broader issues surrounding algorithmic bias in criminal justice.

What is COMPAS?

COMPAS is a risk assessment tool developed by Northpointe (now Equivant) used across the United States to predict the likelihood of a defendant re-offending. It generates scores based on a defendant's answers to a questionnaire, predicting the "Risk of Recidivism" and "Risk of Violent Recidivism." These scores are then used by judges and other officials to inform decisions at various stages of the criminal justice system.

ProPublica's Methodology

To assess the COMPAS algorithm, ProPublica obtained data on over 10,000 criminal defendants in Broward County, Florida. They compared the algorithm's predicted recidivism rates with the actual recidivism rates of defendants over a two-year period. ProPublica's analysis focused on:

• Accuracy: How well COMPAS predicted recidivism overall.
• Bias: Whether the algorithm unfairly discriminated against certain groups, particularly based on race.

Key Findings of the COMPAS Analysis

ProPublica's COMPAS analysis revealed several critical findings:

• Overall Accuracy: The COMPAS algorithm correctly predicted recidivism about 61% of the time, with only 20% accuracy in predicting violent recidivism.
• Racial Disparities:
  • Black defendants were significantly more likely than white defendants to be incorrectly classified as high risk of recidivism. 45% of black defendants who did not re-offend were misclassified as high risk, compared to 23% of white defendants.
  • White defendants were more likely than black defendants to be incorrectly classified as low risk of recidivism. 48% of white defendants who did re-offend were misclassified as low risk, compared to 28% of black defendants.
  • Even when controlling for prior crimes, future recidivism, age, and gender, black defendants were 45% more likely to be assigned higher risk socres than white defendants.
• Disparities in Violent Recidivism: Similar racial disparities were observed in predictions of violent recidivism.

Statistical Analysis and Insights

ProPublica employed various statistical models to explore the data, including logistic regression and Cox proportional hazards models. The results consistently showed that:

• Age: Younger defendants were more likely to be assigned higher risk scores.
• Race: Black defendants were more likely to be assigned higher risk scores, even after controlling for other factors.
• Gender: Surprisingly, female defendants were slightly more likely to receive higher risk scores than men, considering the same factors.

These statistical analyses underscore the complex interplay of factors influencing COMPAS scores, and highlight the potential for unintended bias.

Prior Research

ProPublica's work builds on previous research examining recidivism risk assessment tools. Earlier studies showed the COMPAS score correctly predicted recidivism for black and white defendants at roughly the same rate (59 percent for white defendants, and 63 percent for black defendants) but made mistakes in very different ways.

Implications and the Call for Further Scrutiny

The findings of ProPublica's COMPAS analysis raise significant concerns about the fairness and transparency of algorithmic risk assessments in criminal justic. Eric Holder, former U.S. Attorney General, has also voiced concerns about potential bias in these risk assessment tools.

The use of biased algorithms can perpetuate and exacerbate existing inequalities within the legal system, leading to disproportionate outcomes for certain demographic groups. As algorithms become more prevalent in criminal justice, it is crucial to:

• Conduct rigorous and independent audits of these tools.
• Ensure transparency in the design and implementation of algorithms.
• Develop strategies for mitigating and correcting algorithmic bias.

Conclusion

ProPublica's analysis of the COMPAS algorithm provides a valuable case study of the challenges and pitfalls of using algorithms in criminal justice. While these tools may offer the potential to improve decision-making, it is essential to address issues of bias and ensure that they are used fairly and equitably. Further research, oversight, and public discourse are needed to navigate the ethical and societal implications of algorithmic risk assessments in the legal system.