Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

Recently, I have compared two correlated AUC with the method of Delong. Someone said that since the CI’s overlap, we cannot state the two models were different. I know that the method of Delong calculated correlated AUC, but I don't know how to answer his question in a formal way. Could someone give some help?

share|improve this question

1 Answer

up vote 3 down vote

The overlap of two 95% confidence intervals does not in itself indicate lack of an effect at the 0.05 level. Just touching error bars is usually about 0.01 probability assuming null (depending on the similarity of the bars). You can get an idea of whether the difference between the two AUCs is significant or not if you take the average CI width between the two points and multiply it by sqrt(2). If that is less than the difference then there is an effect at the 0.05 level.

share|improve this answer
Thank John,your method works in assessment of 2 group's means as t-test. I don't know whether your method is applicable in comparing correlated AUC. – Yao Zhu Feb 22 '12 at 3:58
This will be conservative. If you have an effect you're OK with this. It might tend toward Type II errors because they're correlated but if the DeLong really takes into account the correlation then it will be fine. Either it's a confidence interval or it's not. (actually, I might argue that because it takes into account the correlation it's not... :)... but it will still have this inferential property) – John Feb 22 '12 at 8:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.