How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are there functions in R to help with this?
Several ideas and references are discussed in:
- A simple generalization of the area under the ROC curve to multiple class classification problems.
- Multi-class ROC (a tutorial) (using "volumes" under ROC)
Other approaches include computing
- macro-average ROC curves (average per class in a 1-vs-all fashion)
- micro-averaged ROC curves (consider all positives and negatives together as single class)
You can see examples in some libraries like scikit-learn.
See also this other thread in CrossValidated: How to compute precision/recall for multiclass-multilabel classification?
One of the ideas is to use one-vs-all classifier. This answer gives move information about it, including some R code.
Here's a plot from that answer