Thanks for implementing this!
In both delong_roc_variance() and delong_roc_test(), predictions and predictions_one/predictions_two are defined as "probability of being class 1". Are these arguments necessarily arrays of probability values? I think ROC curves can be built off of any kind of numerical 'score', not only probabilities.
Your example code (https://github.com/yandexdataschool/roc_comparison/blob/master/example.py) passes normal random variables, rather than probabilities (variable is even called scores). I also found that scaling the inputs has no effect on the p value returned from delong_roc_test, and there are not errors returned for input values outside [0, 1].
So would it make sense to revise the parameter descriptions to call them 'scores' rather than 'probabilities'? It might even make sense to rename the variable from predictions to scores.
Thanks for implementing this!
In both
delong_roc_variance()anddelong_roc_test(),predictionsandpredictions_one/predictions_twoare defined as "probability of being class 1". Are these arguments necessarily arrays of probability values? I think ROC curves can be built off of any kind of numerical 'score', not only probabilities.Your example code (https://github.com/yandexdataschool/roc_comparison/blob/master/example.py) passes normal random variables, rather than probabilities (variable is even called
scores). I also found that scaling the inputs has no effect on the p value returned fromdelong_roc_test, and there are not errors returned for input values outside [0, 1].So would it make sense to revise the parameter descriptions to call them 'scores' rather than 'probabilities'? It might even make sense to rename the variable from
predictionstoscores.