5
$\begingroup$

I applied two different classifiers against the same validation set. It turns out that classifier A is better than classifier B in terms of ROC curve. However, classifier B is better than classifier A in terms of confusion matrix. How to explain this kind of contradiction?

$\endgroup$
  • 1
    $\begingroup$ Add data and graphs. Otherwise it's difficult to know what you mean. Also, what does "better in terms of confusion matrix" mean? You can order them by accuracy, or f-score, or precision, or recall, or sensitivity, or specificity, .... $\endgroup$ – Stumpy Joe Pete Mar 16 '13 at 22:26
  • $\begingroup$ You still haven't told us what it means to be "better" in terms of confusion matrix. There isn't only one way to compare confusion matrices! $\endgroup$ – Stumpy Joe Pete Mar 17 '13 at 4:14
6
$\begingroup$

A ROC curve shows you performance across a range of different classification thresholds and a confusion matrix only shows you one (typically when $Pr(y > .5)$).

$\endgroup$
  • 1
    $\begingroup$ In general, which metric, ROC or confusion matrix is a more reasonable one? $\endgroup$ – user3125 Mar 17 '13 at 0:13
  • $\begingroup$ The ROC curve subsumes the confusion matrix. It is not as easily interpretable though. $\endgroup$ – Zach Mar 17 '13 at 1:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.