Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, so what is the advantage of adding the ROC graph? What can one tell from it?

share|improve this question
This isn't really any different from lots of cases where the data are described numerically in the text, but people like to present the information visually as well. – gung Mar 30 '14 at 20:59

I usually give the ROC plot but not the AUC: For my applications it is usually clear that either a specific or a sensitive regocognition is needed. The ROC is different for classifiers that are specific but not sensitive vs. sensitive but not specific while the AUC hides this information.

Besides, one can put a whole lot of further information into the plot, e.g. color-coding the thresholds (check whether the classifier is well calibrated if the primary output is posterior probability), model stability (after resampling validation), or confidence regions for a chosen classifier (if one chooses a threshold). Finally, you can even put "extented" measures of sensitivity and specificity which do not require the thresholding @FrankHarrell fights against, e.g. it is possible to extend the concept of sensitivity and specificity and the concept behind Brier's score to yield such measures.

share|improve this answer
Is there something about probabilistic decision making that doesn't work in your situation? – Frank Harrell Mar 31 '14 at 0:11
@FrankHarrell: Well, yes: communication. I spend quite a lot of my working time doing translations: physicist <-> chemist (<-> chemometrician) <-> statistician <-> medical doctor, and I find the art of doing this well has a lot to do with allowing simplifications and finding compromises (and trying to avoid to gross misconceptions). I tackle the "don't throw away information" problem by sneaking in continuous measures (as chemist I prefer MSE/RMSE) which are well (or at least better) behaved, but their names and purpose are familiar. – cbeleites Mar 31 '14 at 14:40
I feel that probabilities can work well if communication is careful or the problem involves sports or card playing. – Frank Harrell Mar 31 '14 at 19:34

There is great value in showing the entire ROC esp. when comparing two different classifiers, as it helps us to see whether different curves cross each other. One is not superior to the other, overall, if they cross - see the Figure 3 (screenshot shown below) from

Seong Ho Park, J. M. G. C.-H. J. (2004). Receiver Operating Characteristic (ROC) Curve: Practical Review for Radiologists. Korean Journal of Radiology, 5(1), 11–18. Crossing ROC curves from Park et al., 2004

In our own study, we noticed this behaviour for biomarkers that have similar performance, and partial AUCs can be studied to demonstrate their utility for particular applications e.g. in high specificity regions for early detection of Alzheimer's disease.

Citation: Raamana, P. R., Weiner, M. W., Wang, L., & Beg, M. F. (2015). Thickness network features for prognostic applications in dementia. Neurobiology of Aging, 36, S91–S102.

Comparison of ROCs from different methods

Hope that helps make the case for presenting the ROC always.

share|improve this answer

The ROC curve is the specificity/sensitivity plot; the AUC is the Area Under Curve. To be brief, the ROC curve can be interesting because it allows comparison of the sensitivity/specificity behaviour of the model. More simply:

$ROC = (x,y) \in R^2 \Rightarrow AUC = z$ but $AUC = z \nRightarrow ROC = (x,y) \in R^2$

share|improve this answer
I still don't get it. What could one use the ROC graph for if he only needs the AUC to really evaluate the performance? – Jack Twain Mar 30 '14 at 18:08

I have not seen a single example where the graph changes our actions or the way we think. I think the ink:information ratio in an ROC graph is enormous. But the worst thing about it is that it tempts us to try to select a cutoff for the predicted risk, which is arbitrary, and inconsistent with optimum decision making.

Beware of the word classifier which implies discarding continuous information.

share|improve this answer
Temptation for a cutoff is a point, but OTOH giving the ROC IMHO is already much better than reporting sensitivity and specificity at some arbitrary default cutpoint without any further information. – cbeleites Mar 30 '14 at 22:54
Only if sensitivity and specificity are helpful. I find them not to be. If we were regressing age on blood pressure we'd never create an ROC curve. Why we do it when $Y$ is binary is beyond me. – Frank Harrell Mar 30 '14 at 23:34
@FrankHarrell regression doesn't help when you need to make a decision, rather than a prediction. At some point you need to define a threshold. You can't treat a patient for 70%. In those settings, ROC curves provide a valuable summary of the model at different operating settings (in part due to the fact that ROC curves are understood by many). That said, I agree on the high ink:information ratio. – Marc Claesen Mar 31 '14 at 4:57
I have to disagree [and medical diagnosis is one of my main areas of research]. There are only two situations I can think of where your reasoning applies: (1) when the analyst is the possessor of the utility function (highly unusual) or (2) when decisions must be made instantly without the time to assess utilities. Not only are probabilities the optimum inputs to true decision makers, they allow one to make no decision, when more data are needed. – Frank Harrell Mar 31 '14 at 12:18
@FrankHarrell: I totally agree that lots of valuable information are lost with the dichotomization. As chemist, my primary philosophical concept of the world is quantitative (substances come at concentrations). So qualitative analysis ("native chemical" version of classification) is the result of comparing a primarily quantitative result to a threshold. However, there are situations which are more basically grouped than continuous. Few people are 20% pregnant... And even in some fundametally continuous situations it just doesn't make sense to cry over the threshold: (...) – cbeleites Mar 31 '14 at 14:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.