Problem in interpreting AUC confidence interval

Question

I calculated AUC using SPSS and the output was: 0.623, std.Error=0.056. p=0.055. lower Bound was 0.513 and upper was 0.743

How could it be that the p value is not significant and the lower Bound of C.I was more than 0.5?

Answer 1

This is because there are multiple ways to test the AUC, and the p-value does not correspond to the CI.

The confidence interval you obtained is not an exact CI but an approximation. Furthermore, it's a less efficient one at that because it's transformed onto a non-natural scale for the statistic in question. Analogously, confidence intervals for binary proportions are more efficient when they are on the log odds scale which is the natural parametrization. The only difficulty is not having symmetric CIs when back-transformed.

In this case, the CI approximation would be said to be a "anti-conservative" one because the CI would show that the AUC is statistically significant whereas the actual test which SPSS applies does not.

SPSS is frustratingly bereft of documentation for how they compute this $p$-value. But I'd be willing to bet money the $p$-value comes from the Mann-Whitney test. The U statistic which is tested by Mann-Whitney has a direct correspondence to the interpretation of the AUC: the chance that any case and any control drawn at random have the case's assigned probability greater than the control's.

Arguably, that means that the test applied here is a good one. The $p$-value is the appropriate "decision" you should make about the utility of the test, which is to say your classifier is not better than chance alone.

If you wanted the CI to agree with the p-value there are very advanced techniques of inverting statistical tests to obtain confidence intervals, and/or bootstrapping, but I would dissuade you from trying since it won't tell you anything you don't already know.

Answer 2

For the area under the receiver operating curve, the null hypothesis is an AUC of 0.5. The p-value would only be significant (at alpha 0.05) if the 95% CI doesn't cross 0.5.