I have been analyzing the accuracy of 3 prognostication scores in predicting a certain binary outcome using ROC curves and significance testing for differences in AUCs between the curves (a figure of the ROC curves and the AUCs + 95% confidence intervals for each score is in the post).
As you can see from the figure, Score A has the lowest AUC (0.75). When comparing the AUC for score A against score B, score B's AUC is significantly greater (p=0.02). However, despite score C having a slightly higher AUC and a tighter confidence interval than score B, score C's AUC is not significantly higher than score A's (p=0.08). I was really confused as to why score C's difference would not be significant, given these factors and that all these tests are conducted on the identical dataset (there is no missing data). I am not well versed with the mathematics behind significance testing to compare AUCs and was wondering if there was something that might explain this peculiar trend?
AUC of score A vs. score C: No difference (p=0.08)
I have found these same results across multiple packages (ex. rocgold() on Stata and the pROC package on R) and different methods (boostrap, delong, etc.) and have gotten the same results across all of these approaches. I am extremely curious why I have been finding these results. Thank you so much!