# ROC for more than 2 outcome categories

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are there functions in R to help with this?

-
Do you mean how to construct ROC's when there are +2 models? –  user30490 Aug 18 '14 at 21:23
Or do you mean that there are 4 outcome categories? –  gung Aug 18 '14 at 21:46
Categories :) I edited my post –  Marcin Kosinski Aug 18 '14 at 21:51
I would suggest checking out this answer: stats.stackexchange.com/questions/38541/… –  user30490 Aug 18 '14 at 21:58
What about ROC curves makes them insightful to you? Are you really interested in concordance probabilities ($c$-index; ROC area; pure discrimination measure)? I find the ROC area to be helpful even though the curves are not helpful to me. And you can generalize the idea of concordance probability to multiple categories using Somers' $D_{xy}$ rank correlation coefficient. –  Frank Harrell Aug 18 '14 at 21:59

Several ideas and references are discussed in:

Other approaches include computing

• macro-average ROC curves (average per class in a 1-vs-all fashion)
• micro-averaged ROC curves (consider all positives and negatives together as single class)

You can see examples in some libraries like scikit-learn.

-
FYI - the link to the multi-class ROC tutorial doesn't work –  rocinante Aug 18 '14 at 22:04
@rocinante Fixed. –  Josh Aug 18 '14 at 22:05
@Josh that's the vast and outstanding piece of literature :) Thank you very much. That was something I was looking for! CV is a great place. –  Marcin Kosinski Aug 18 '14 at 22:28

One of the ideas is to use one-vs-all classifier. This answer gives move information about it, including some R code.

Here's a plot from that answer

-
Very usefull comment. Thanks for that! Nice idea by the way. –  Marcin Kosinski Aug 19 '14 at 13:50