ROC and accuracy results: how can AUC be one if the classifier has made mistakes?

Question

Trying to get the accuracy and the ROC curve with R (mlr package) I get the following results:

Absolute confusion matrix:
        predicted
true     CHAV PMP -err.-
  CHAV      4   0      0
  PMP       2   5      2
  -err.-    2   0      2

predicted
true   CHAV      PMP                       
  CHAV 1         0      tpr: 1    fnr: 0   
  PMP  0.29      0.71   fpr: 0.29 tnr: 0.71
       ppv: 0.67 for: 0 lrp: 3.5  acc: 0.82
       fdr: 0.33 npv: 1 lrm: 0    dor: Inf 

It is clear that the model has two mistakes (82% accurracy)

But when I try to calculate the AUC, I get the following result:

auc 
  1 

I don't understand why the AUC is 1, even though the classifier has made 2 mistakes.

Answer 1

A confusion matrix measures the performance of your classifier in a given threshold, typically at 0.5 if no other value is specified. AUC measures the ranking of all your scores at no specific threshold, just considers the order of all your predictions. If all your positive examples have higher probability than your negative examples but some of your negatives have higher probability than 0.5 that could cause what you describe. AUC has on x-axis FPR, false positive rate, and on y-axis TPR. In the scenario I describe there is a pointin which you correctly identify all your positives without making any mistakes. TPR=1 and FPR=0. That drives the curve to the top left corner and gives AUC=1.