1
$\begingroup$

I've made several predictive models, tuned on maximizing the AUC score. The models have an AUC score of 0.81 and 0.57.

I wanted to see the predictions from the models in a confusion matrix, so I used the same models and predicted on the same testset as I did by the AUC score.

The confusion matrix showed that the models took the majority class as prediction.

How is it possible that the AUC score's are not the same?

Intuitively I would expect that the AUC scores should be the same, entirely if I look at the confusion matrix that predicts exactly the same majority class for both models.

$\endgroup$
2
$\begingroup$

AUC stands for area under the curve. It is the integral of the correct choices at all different "cutoff" values. The confusion matrix is taken at one particular cutoff value.

In your case, you say that the confusion matrices labeled all cases as the majority case. That just means that all of the models you ran thought the best cutoff was to do this (there are ways to change the default value of the cutoff) but that there were differences at other cutoff levels.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! So if I understand it correctly it's possible, because AUC is an collection off the "cutoff" values and the confusion matrix is the best cutoff (single value). $\endgroup$ – Arnand May 9 '17 at 10:46
  • $\begingroup$ Yes. If you look at the ROC curve, the confusion matrix is at a single point, the AUC is the area between the ROC curve and a 45 degree line. $\endgroup$ – Peter Flom May 9 '17 at 10:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.