1
$\begingroup$

I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated

$\endgroup$
1
$\begingroup$

Accuracy is equivalent to AUC for models making binary predictions (although accuracy gives you a more direct interpretation).

In the case you model makes continuous predictions, a ROC curve will allow you to choose a cut-off on which to compute accuracy. In this case, both are complementary, but in the end which metrics to use depends on:

  • Are you going to set a cut-off in your predictions anyway, and report only a positive/negative prediction? Then use ROC to determine the cut-off and compute accuracy on it;
  • Are you interested to know if the predictions are different in one group than an other, and want to report a probability of the data point being positive? In this case, use AUC.
$\endgroup$
  • $\begingroup$ @Siga: do you need any more help with your question? $\endgroup$ – Calimo Jan 18 '14 at 9:03
  • $\begingroup$ not really. The answer is quite helpful. But I thought calculating accuracy meant the calculation using confusion matrix. Thank you very much for asking :) $\endgroup$ – Siga Jan 19 '14 at 10:50
  • $\begingroup$ The accuracy is implicitly computed from the confusion matrix, so does a ROC curve (with multiple confusion matrices, one by cut-off level). $\endgroup$ – Calimo Jan 19 '14 at 10:53
  • $\begingroup$ So accuracy is better to be used than AUC when one wants to show only positive/negative prediction? $\endgroup$ – Siga Jan 19 '14 at 10:57
  • $\begingroup$ I guess it's a matter of taste, as I said accuracy and AUC are equivalent in this case, but accuracy allows a direct interpretation of the misclassification rate. $\endgroup$ – Calimo Jan 19 '14 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.