I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
Accuracy is equivalent to AUC for models making binary predictions (although accuracy gives you a more direct interpretation).
In the case you model makes continuous predictions, a ROC curve will allow you to choose a cut-off on which to compute accuracy. In this case, both are complementary, but in the end which metrics to use depends on:
- Are you going to set a cut-off in your predictions anyway, and report only a positive/negative prediction? Then use ROC to determine the cut-off and compute accuracy on it;
- Are you interested to know if the predictions are different in one group than an other, and want to report a probability of the data point being positive? In this case, use AUC.