I am trying to evaluate a multi-class classification model, say Random Forest, I built in R. So far, my approach has been looking at the
ConfusionMatrix from the
caret package and other results that are calculated by that function, but it gives me only a class wise result. I am also calculating micro and macro averages for estimating the overall accuracy for the model.
Now, when I am trying to plot the ROC curve, I have two options:
- One-vs-One approach: gives me nC2 combinations of ROC curves, which I am not sure how to interpret. The pROC package implements this method, suggested by Hand and Till because it supposedly gives a more accurate AUC.
- One-vs-All approach: gives me n ROC curves and their corresponding AUC. A reference for this I found in another similar question
Is there any preferred method for plotting the ROC in this scenario? If so, why?
And can this choice of it being either One-vs-One or One-vs-All be generalized to any multi-class model?