# Selecting a loss-function for k-fold cross-validation over shrinkage parameter

I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $\lambda$ on the basis of cross-validation (CV).

In this case, I have 50k observations and my outcome variable has three (ordered) levels.

Three options occur to me:

1. I normally see people using CV to pick a $\lambda$ by minimizing mis-classifcation error.
2. I could also pick $\lambda$ in CV to maximize the joint log likelihood across folds.
3. I could maximize the Spearman rank-correlation of the predicted class with the outcome.

Question: What are the considerations involved in picking among these cross-validation loss functions (a) in the ordinal case and (b) in the general multinomial case (for #1 and #2)?

Minimizing misclassification error seems most intuitive to me at first glance, but perhaps I'm missing something. Maybe something about bias/variance is going on here -- one answer here indicates that k-fold cross-validation is subject to substantial imprecision in general, but I don't know what the implications are for picking the cross-validation loss function.

• Maybe this question helps. In my experience, only deviance works well, as it is stated in the accepted answer. Hope it helps. – lrnzcig Apr 26 '16 at 9:11