Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are there functions in R to help with this?

share|improve this question
1  
Do you mean how to construct ROC's when there are +2 models? –  user30490 Aug 18 at 21:23
    
Or do you mean that there are 4 outcome categories? –  gung Aug 18 at 21:46
    
Categories :) I edited my post –  Marcin Kosinski Aug 18 at 21:51
    
I would suggest checking out this answer: stats.stackexchange.com/questions/38541/… –  user30490 Aug 18 at 21:58
4  
What about ROC curves makes them insightful to you? Are you really interested in concordance probabilities ($c$-index; ROC area; pure discrimination measure)? I find the ROC area to be helpful even though the curves are not helpful to me. And you can generalize the idea of concordance probability to multiple categories using Somers' $D_{xy}$ rank correlation coefficient. –  Frank Harrell Aug 18 at 21:59

2 Answers 2

up vote 4 down vote accepted

Several ideas and references are discussed in:

Other approaches include computing

  • macro-average ROC curves (average per class in a 1-vs-all fashion)
  • micro-averaged ROC curves (consider all positives and negatives together as single class)

You can see examples in some libraries like scikit-learn.

See also this other thread in CrossValidated: How to compute precision/recall for multiclass-multilabel classification?

share|improve this answer
    
FYI - the link to the multi-class ROC tutorial doesn't work –  rocinante Aug 18 at 22:04
    
@rocinante Fixed. –  Josh Aug 18 at 22:05
    
@Josh that's the vast and outstanding piece of literature :) Thank you very much. That was something I was looking for! CV is a great place. –  Marcin Kosinski Aug 18 at 22:28

One of the ideas is to use one-vs-all classifier. This answer gives move information about it, including some R code.

Here's a plot from that answer

enter image description here

share|improve this answer
    
Very usefull comment. Thanks for that! Nice idea by the way. –  Marcin Kosinski Aug 19 at 13:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.