Creating ROC curve for multi-level logistic regression model in R

Question

I used the functions from this link for creating ROC curve for logistic regression model. Since the object produced by glmer in lme4 package is a S4 object (as far as I know) and the function from the link cannot handle it.

I wonder if there are similar functions for creating ROC curve for multi-level logistic regression model in R.

Answer 1

There's a whole lot of literature about multi-class extensions for ROC.

I have some presentations with illustrations how the calculation works at softclassval's home page (softclassval calculates sensitivities etc. if you have partial class memberships, also for multiple classes - but that is probably an overkill for your problem).

For sensitivity and specificity, the spelled out definitions lead to a very straightforward extension:

• sensitivity: what proportion of truly class $c$ cases are correctly recognized by the model?
• specificity: what proportion of cases truly not belonging to class $c$ are correctly recognized as not coming from class $c$?

If you think about medical diagnostics/epidemiology, the set up is always multinomial from a philosophical point of view: the normal/healthy/control group in fact is rather a "not this disease" group which may contain a whole lot of other diseases. Sometimes classes are mutually exclusive, more often they are not (having, say, a brain tumour does not mean that you cannot have hepatitis nor does it save you from breaking your arm)

• I use package ROCR to plot ROCs, but there are plenty alternatives in R (e.g. pROC - pROC's home page has a comparison of several R packages dealing with ROC generation in R).

Answer 2

Not possible.

The very idea of ROC requires the concept of sensitivity and specificity, which in turn take only real numbers. To have the idea of ROC working with more than two-valued logic, you would need to accept that sensitivity and specificity are vectors.

You might always convert your dependent variable into set two-level dummy variables and perform a series of ROCs. But I guess it's not what you are looking for.