Why is the area under the ROC curve better than raw accuracy as an out-of- sample evaluation metric?
2
-
2$\begingroup$ It's not, it depends. Especially if you have imbalanced classes. $\endgroup$– user2974951Aug 13, 2019 at 13:05
-
$\begingroup$ I think you will find the information you need in the linked thread. Please read it. If it isn't what you want / you still have a question afterwards, come back here & edit your question to state what you learned & what you still need to know. Then we can provide the information you need without just duplicating material elsewhere that already didn't help you. $\endgroup$– gung - Reinstate MonicaAug 13, 2019 at 14:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In a short answer we could say the following. It’s not better per se, but what we can say more correctly is that it is a general representation of wrong/right classification percentages of the model for any given cutoff chosen. As such it allows to evaluate the discriminatory power of the model regardless the choice of a specific cutoff.
The representation of the model accuracy via contingency tables (and the related statistics) depends on the chosen cutoff, based on which, in a binary classification model, you can draw a contingency table with the percentages of true positives, false positives, true negatives and false negatives. So for your model, you choose a cutoff and, based on that, you draw a table showing the 1-specificity and sensitivity of the model (and their complements to 1).
On the contrary the ROC curve plots the same concepts of 1-specificity ($x_{i}$) and sensitivity ($y_{i}$) for each given cutoff i on a $(x_{i},y_{i})$ coordinate system assuming that for each point in the plot the cutoff varies (so each point in the graph represents de facto the results in the contingency table obtained assuming that the cutoff is i). As you can see it can be considered a generalization of the contingency table approach for a generic i that is allowed to vary in the ROC plot. Therefore it allows to analyze the classification performance of the model (and the trade-off of specificity vs sensitivity) regardless the specific cutoff that you choose, thus segregating the two distinct important problems of choosing a good classification model per se (for any give cutoff), and then choosing a good cutoff. The AUC is just a numerical representation of the area under the ROC allowing to put the overall performance described by the ROC in a synthetic numerical form.
