I used the functions from this link for creating ROC curve for logistic regression model. Since the object produced by glmer in lme4 package is a S4 object (as far as I know) and the function from the link cannot handle it.
I wonder if there are similar functions for creating ROC curve for multi-level logistic regression model in R.
There's a whole lot of literature about multi-class extensions for ROC.
I have some presentations with illustrations how the calculation works at softclassval's home page (softclassval calculates sensitivities etc. if you have partial class memberships, also for multiple classes - but that is probably an overkill for your problem).
For sensitivity and specificity, the spelled out definitions lead to a very straightforward extension:
If you think about medical diagnostics/epidemiology, the set up is always multinomial from a philosophical point of view: the normal/healthy/control group in fact is rather a "not this disease" group which may contain a whole lot of other diseases. Sometimes classes are mutually exclusive, more often they are not (having, say, a brain tumour does not mean that you cannot have hepatitis nor does it save you from breaking your arm)
update: @Adam
I know this paper:
which deals with independent classes. Basically with $n$ independent classes, you get an $n-1$ dimensional "surface" in $n$ dimensions spanned by the e.g. sensitivity for each class.
Here's something about ordered levels:
But I cannot access it, so I can't tell you anything further.
Not possible.
The very idea of ROC requires the concept of sensitivity and specificity, which in turn take only real numbers. To have the idea of ROC working with more than two-valued logic, you would need to accept that sensitivity and specificity are vectors.
You might always convert your dependent variable into set two-level dummy variables and perform a series of ROCs. But I guess it's not what you are looking for.
This question is super old, but for those coming to it now I believe the author is purposely referring to multi-level models, not multi-class models.
Some info regarding multi-level models:
https://www.pymc.io/projects/examples/en/latest/generalized_linear_models/multilevel_modeling.html
https://www.bristol.ac.uk/cmm/learning/videos/random-intercepts.html
With regard to my interpretation of the original question, I'll look for a reference, but here is my crack at it.
We can note that the purpose of a multilevel model is not to generate a 'longitudinal model' but to be more specific about the structure of our data when we define the model, such that the model's coefficient estimates take into consideration the 'precision' information contained in the repeated measures. With that in mind, the real issue with using a ROC curve to evaluate a multi-level model is that, if your model is to be deployed beyond the training, predictions should be made using the average slope/intercept learned by the model, or with new per group data w/ per group slope/intercept, not the same data used to generate the per-group slope/intercept. If you test on the same dataset you trained on, predicting using the per-group slope/intercept, the ROC will the performance in the training data, which may not be what you are looking to evaluate. If you would like to evaluate the performance of the model within the training set, it might be better to examine the magnitude of imprecision in the estimated parameters (see the pymc link above), and/or take a look at the model's PCC (https://www.pymc.io/projects/docs/en/stable/learn/core_notebooks/posterior_predictive.html).
