I have read in several places that R Squared is not an ideal measure when a model is fit using LASSO. However, I'm not clear on exactly why that is.
In addition, could you recommend the best alternative?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityThe goal of using LASSO is obtaining a sparse representation (of a predicted quantity) in the sense of not having many covariates. Comparing models with $R^2$ tends to favor models with lots of covariates: in fact, adding covariates unrelated to the outcome will never decrease $R^2$ and almost always increases it at least a little bit. The LASSO model will identify the model with the optimal penalized log-likelihood (an unpenalized log-likelihood is monotonically related to the $R^2$). Validation statistics that are more widely used to compare LASSO models to other types of models are, for instance, the BIC or cross-validated $R^2$.