I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
Accuracy is equivalent to AUC for models making binary predictions (although accuracy gives you a more direct interpretation).
In the case you model makes continuous predictions, a ROC curve will allow you to choose a cut-off on which to compute accuracy. In this case, both are complementary, but in the end which metrics to use depends on:
- Are you going to set a cut-off in your predictions anyway, and report only a positive/negative prediction? Then use ROC to determine the cut-off and compute accuracy on it;
- Are you interested to know if the predictions are different in one group than an other, and want to report a probability of the data point being positive? In this case, use AUC.
-
$\begingroup$ @Siga: do you need any more help with your question? $\endgroup$– CalimoCommented Jan 18, 2014 at 9:03
-
$\begingroup$ not really. The answer is quite helpful. But I thought calculating accuracy meant the calculation using confusion matrix. Thank you very much for asking :) $\endgroup$– SigaCommented Jan 19, 2014 at 10:50
-
$\begingroup$ The accuracy is implicitly computed from the confusion matrix, so does a ROC curve (with multiple confusion matrices, one by cut-off level). $\endgroup$– CalimoCommented Jan 19, 2014 at 10:53
-
$\begingroup$ So accuracy is better to be used than AUC when one wants to show only positive/negative prediction? $\endgroup$– SigaCommented Jan 19, 2014 at 10:57
-
$\begingroup$ I guess it's a matter of taste, as I said accuracy and AUC are equivalent in this case, but accuracy allows a direct interpretation of the misclassification rate. $\endgroup$– CalimoCommented Jan 19, 2014 at 10:59