4
$\begingroup$

For one of my classes, the teacher asked this question of when the AUC is always equal to 0. He provided hints that it has something to do with the Wilcoxon Rank Sum test and how the AUC is actually calculated (IE: trapezoidal). I have been searching high and low and cannot fathom a situation where the AUC would always be equal to 0. Does anyone have any examples or sources that might point me in the right direction?

Thank you!

$\endgroup$

closed as off-topic by Sycorax, SmallChess, Michael Chernick, mdewey, kjetil b halvorsen Apr 19 '17 at 12:19

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Self-study questions (including textbook exercises, old exam papers, and homework) that seek to understand the concepts are welcome, but those that demand a solution need to indicate clearly at what step help or advice are needed. For help writing a good self-study question, please visit the meta pages." – Sycorax, SmallChess, mdewey, kjetil b halvorsen
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ Please add the [self-study] tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$ – gung Apr 18 '17 at 16:04
3
$\begingroup$

When you make "perfectly" wrong prediction, i.e., reverse of ground truth.

> y=sample(0:1,1000,replace = T)
> y_hat=ifelse(y==1,0,1)
> caret::confusionMatrix(y_hat,y)

Confusion Matrix and Statistics
          Reference
Prediction   0   1
         0   0 485
         1 515   0

               Accuracy : 0          
                 95% CI : (0, 0.0037)
    No Information Rate : 0.515      
    P-Value [Acc > NIR] : 1.0000     

                  Kappa : -0.9982    
 Mcnemar's Test P-Value : 0.3591     

            Sensitivity : 0.000      
            Specificity : 0.000      
         Pos Pred Value : 0.000      
         Neg Pred Value : 0.000      
             Prevalence : 0.515      
         Detection Rate : 0.000      
   Detection Prevalence : 0.485      
      Balanced Accuracy : 0.000      

       'Positive' Class : 0       


> pred=ROCR::prediction(y_hat,y)
> perf_AUC=ROCR::performance(pred,"auc")
> AUC=perf_AUC@y.values[[1]]

> AUC
[1] 0
$\endgroup$
  • $\begingroup$ Thanks for the example! Is this inaccurate prediction based on an applied model? I'm just a bit confused because my professor suggested that this would be an inherent problem with how the AUC is calculated rather than a bad model. As in, if there were an AUC of 0.3, the answer would be to reverse your models predictions. In his example though, flipping the model would still result in an AUC that is worse than random guessing. $\endgroup$ – strucktea Apr 18 '17 at 22:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.