Cross validation for lasso logistic regression

Question

I am writing a routine for logistic regression with lasso in matlab. So the problem is to minimize the negative log-likelihood function with the penalty term

$$\sum \left(\log(1 + e^{X_i' \beta}) - y_i X_i' \beta\right) + \lambda \sum |\beta_i|$$

where $\beta$ is the model parameter, $X_i$ is the $i$th row of matrix $X$, and $y_i$ is the value of observation $i$.

My first question is for a 5-fold cross-validation, which criterion should I use to pick the best value of $\lambda$? Should I use the value of the logit function on the validating data set or mis-classification rate on the validating data?

Answer 1

The short answer is, its up to you, depending on your interest. In the past I have used AIC for Lasso.

However it sounds like you are using this model for prediction, and thus using the mis-classification rate is a good idea. However misclassification can be categorized in many ways. Are you interested in the the absolute % classified correctly? Or maybe you just care about of those classified as 1 (or yes, etc), how many of those were classified correctly? I would do some reading into Positive Predictive values, Negative predictive values, etc.

https://en.wikipedia.org/wiki/Positive_and_negative_predictive_values

In addition when doing your cross validation, there are a plethora of criteria you could use to validate your model. A short list of other common criterion are:

• $R^2$
• $MSE$
• $Mallow’s$ $C_p$
• $AIC$

Look them up and see which is most relevant to you!