For example, the author of this very article has done such an analysis. Here's her R notebook:
https://github.com/propublica/compas-analysis/blob/master/Co...
Her analysis shows (within the limitations of the frequentist paradigm) that:
a) the predictor is useful - score_factorHigh and score_factorMedium both have p-values that are essentially zero.
b) The predictor is not racially biased that much - race_factorAfrican-American:score_factorHigh and the other bias terms have p-values that are > 0.05 .
Look, I'd love it if we required such algorithms to be open source. I'm a huge proponent of both open science and open government. Nevertheless, there is an entire discipline devoted to evaluating predictive algorithms without needing to care about their details - it's called "machine learning".
The wonderful thing about statistics is that even a highly biased person (such as the author of this article) can still reach a correct conclusion that goes against their biases.
People talk about misuse of p-values, but this takes the cake.
Also, this analysis is certainly a useful addition to the literature on this system, but it's one analysis, and regardless of your philosophical stance on p-values, a p-value of .057 in the presence of multiple testing isn't the most convincing thing.
Having said that, the use of non-open predictive systems is a problem for criminal settings. Maybe this thing is biased, but the only way to find out and fix it is to do these sorts of analyses and have this sort of discussion.
The problem isn't the use of prediction systems, it's the use of them without open academic scrutiny, without correcting any biases that emerge.
