# how to compare the AUCs of ROC from nested models and non-nested models from published papers

I'm interested in comparing two AUCs from ROC in two nested models and two AUCs in non-nested models but using the same dataset in a paper. I know DeLong test can be used to compare them, but the formula is kind of complicated for me to do it manually. Also, from the paper, only the mean and the standard error of the AUC are known. Does anyone know how to do it without involving too much calculation or any packages can be used to achieve this?

• 0. Welcome to CV.SE. 1. Unfortunately, this isn't enough information to calculate a DeLong (like) test. 2. I give a very rough approximation we can do if we really have to get something out but I would generally recommend against it.. Commented Jun 20, 2022 at 2:33

The DeLong test requires the AUC-ROC covariance between the models A and B. If we are willing to accept that the covariance between the two models' AUC-ROC is 0, which is a down-right unreasonable assumption in this case, we can get our $$z$$-score simply as: $$z = \frac{\text{AUC}(\text{Model}_A) - \text{AUC}(\text{Model}_B)}{\sqrt{Var\{\text{AUC}(\text{Model}_A)\} +Var\{\text{AUC}(\text{Model}_B)\} }}$$. I would not recommend doing this but hey, it is the best we can hope for given the lack of overall information.
Do note that the covariance would come into the denominator of the calculation shown above, so the (biased) $$z$$-score shown above is slightly more conservative (i.e. give lower $$z$$ scores) than what we would expect if the covariance was not $$0$$. The correct formula would be: $$z = \frac{\text{AUC}(\text{Model}_A) - \text{AUC}(\text{Model}_B)}{\sqrt{Var\{\text{AUC}(\text{Model}_A)\} +Var\{\text{AUC}(\text{Model}_B)\} - 2Covar\{ \text{AUC}(\text{Model}_A), \text{AUC}(\text{Model}_B) \} }}$$.