Cathy O’Neil, a self-proclaimed maths nerd and author of “Weapons of Maths Destruction” explains how police data leads to bias in the criminal justice system. Following is a transcript of the video.
O’NEIL: We don’t actually collect data on crime, we collect the data that the police collect. I’m Cathy O’Neil. I’m a maths nerd, data scientist, and author. When you think about algorithms in the criminal justice system, you have to really think about the data and how the data is built.
So the way predictive policing works is they take the data, they look for crime data and they really don’t have crime data so they use … their best proxy for it which is usually arrest data which means that police are basically sent back to the same neighbourhood where they’re already over policing. And in particular they’re not sent to neighbourhoods that have crime but aren’t — those crimes aren’t found. Now if you think about what that means for the algorithms where you’re looking for crimes based on the location of previous arrests, or previous convictions, or even previous reported crime, that kind of algorithm is intrinsically biased.
And then there’s another kind of algorithm that is a little downstream from the predictive policing algorithm. It’s called the recidivism risk algorithm. Recidivism risk algorithms are used by judges to determine how long to sentence a defendant. And the higher risk of recidivism, which is the risk of returning to prison sometime in the future, or even just getting arrested in the future, the higher risk, the longer someone gets sentenced And what ProPublica found was the compass model which is one version of a recidivism model made mistakes by sending people to prison longer, that kind of mistake, twice as often for African-American defendants as for white defendants, at least in Broward County Florida.
And if — there’s another kind of mistake you can make which is: you look like you’re not coming back, you look low-risk but you actually do come back that kind of risk that kind of mistake was made twice as often for white defendants as for African-American defendants.