Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are there functions in R to help with this?

share|improve this question
Do you mean how to construct ROC's when there are +2 models? – user30490 Aug 18 '14 at 21:23
Or do you mean that there are 4 outcome categories? – gung Aug 18 '14 at 21:46
Categories :) I edited my post – Marcin Kosiński Aug 18 '14 at 21:51
I would suggest checking out this answer:… – user30490 Aug 18 '14 at 21:58
What about ROC curves makes them insightful to you? Are you really interested in concordance probabilities ($c$-index; ROC area; pure discrimination measure)? I find the ROC area to be helpful even though the curves are not helpful to me. And you can generalize the idea of concordance probability to multiple categories using Somers' $D_{xy}$ rank correlation coefficient. – Frank Harrell Aug 18 '14 at 21:59
up vote 7 down vote accepted

Several ideas and references are discussed in:

Other approaches include computing

  • macro-average ROC curves (average per class in a 1-vs-all fashion)
  • micro-averaged ROC curves (consider all positives and negatives together as single class)

You can see examples in some libraries like scikit-learn.

See also this other thread in CrossValidated: How to compute precision/recall for multiclass-multilabel classification?

share|improve this answer
FYI - the link to the multi-class ROC tutorial doesn't work – rocinante Aug 18 '14 at 22:04
@rocinante Fixed. – Josh Aug 18 '14 at 22:05
@Josh that's the vast and outstanding piece of literature :) Thank you very much. That was something I was looking for! CV is a great place. – Marcin Kosiński Aug 18 '14 at 22:28

One of the ideas is to use one-vs-all classifier. This answer gives move information about it, including some R code.

Here's a plot from that answer

enter image description here

share|improve this answer
Very usefull comment. Thanks for that! Nice idea by the way. – Marcin Kosiński Aug 19 '14 at 13:50

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.