AUC in ordinal logistic regression I'm using 2 kind of logistic regression - one is the simple type, for binary classification, and the other is ordinal logistic regression. For calculating the accuracy of the first, I used cross-validation, where I computed the AUC for each fold and than calculated the mean AUC.
How can I do it for the ordinal logistic regression? I've heard about generalized ROC for multi-class  predictors, but I'm not sure how to compute it.
Thanks!
 A: I only like the area under the ROC curve ($c$-index) because it happens to be a concordance probability.  $c$ is a building block of rank correlation coefficients.  For example, Somers' $D_{xy} = 2\times (c - \frac{1}{2})$.  For ordinal $Y$, $D_{xy}$ is an excellent measure of predictive discrimination, and the R rms package provides easy ways to get bootstrap overfitting-corrected estimates of $D_{xy}$.  You can backsolve for a generalized $c$-index (generalized AUROC).  There are reasons not to consider each level of $Y$ separately because this does not exploit the ordinal nature of $Y$.
In rms there are two functions for ordinal regression: lrm and orm, the latter handling continuous $Y$ and providing more distribution families (link functions) than proportional odds.
A: AUC  for ordinal regression is something tricky. You  might want to calculate the AUC for each class by creating dummies to  take value 1 for the class you are calculating the AUC and 0 for the rest of the other classes. If you have 4 classes then you will create 4 AUCs and plot them on the same graph. The main problem with this method is the fact that it penalizes miss-classification equally. Much more intuitively miss-classifying a class 1 into class 3 should be worst than miss-classifying class 1 into class 2.  
