2
$\begingroup$

When reading this article, I noticed that the legend in Figure 3 gives a p-value for each AUC (Area Under the Curve) from the ROC (Receiver Operator Characteristic) curves. It says:

The area under the curve (AUC) is 1.0 (p < .001) for the overall D-IRAP scores, 0.95 (p < .001) for the female picture bias scores and 0.94 (p < .001) for the male picture bias scores.

I'm helping a friend who is developing a similar study. To draw the ROC curves and calculate their AUC, I'm using the R pROC package, which has a function called roc() which returns a roc object. Analyzing this roc object, however, I couldn't find anything similar to a p-value. What am I missing? How did the authors of the cited article come to that p-value? I couldn't find anything in their methods. Does ROC curves usually come with a p-value?

$\endgroup$
  • $\begingroup$ You won't usually find p-values with AUROC estimates, it is not really common practice, so you should delve deeper into the article to see how they did it. Possible solutions: simulations or a statistical test (for ex. proportion test). $\endgroup$ – user2974951 Jan 10 at 7:40
2
$\begingroup$

No, it doesn't. p-values from ROC areas are new to me, but I read the following:

Mason and Graham (2002) show that the ROC area is equivalent to the Mann–Whitney U-statistic testing the significance of forecast event probabilities for cases where events actually occurred with those where events did not occur. Function roc.area() of R-package verification applies this approach and uses function wilcox.test() to calculate the p-value. To me, it looks like this was done in the paper you referenced.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.