How Can I use some variables selected by LASSO?

Question

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice.

I have some data set.

X = 500 x 100 (500 observations x 100 predictor variables)

Y = 500 x 1 (500 response variables)

With that data, I first estimated mean square error (MSE) using Ridge method with 10 fold cross validation. Then, I wanted to find the subset of predictor variables (sparse solution) that might have more impact on the estimation. For this, I used LASSO method with also 10 fold cross validation, and I picked up the best coefficient set that gave me the least MSE value.

At this point, can I directly compare the least MSE value estimated by LASSO method and the MSE value estimated by Ridge method? If it is reasonable and if the MSE value estimated by Lasso method is smaller than the MSE value estimated by Ridge method, can I say that the new model constructed by LASSO method would be better than using the one constructed by the whole predictor variables (Ridge method)?

I have another option in mind. How about applying Ridge method again to the predictor variables selected LASSO method to compare the performance between the subset and whole predictor variables? I don't know this is reasonable or not.

If you know some papers related to my questions, please let me know.

Answer 1

It sounds like you want to use the Elastic Net instead of combining the LASSO and Ridge methods one after the other. The Elastic Net combines the penalty of the LASSO and ridge simultaneously. In R you can do this through the glmnet package. I haven't used it myself but I would assume this would do 10 fold cross validation for you. In general the Elastic Net should give you some of the variable selection properties of the LASSO, while giving the better predictive accuracy normally associated with the ridge method

Sorry forgot to answer the first part of your question. Yes, I don't see why you couldn't compare the MSE estimates if you have used cross validation