Lasso regression coefficients values

Question

I applied lasso regression for variable selection, and out of 10 variables lasso selected 4 variables.

fit.lasso=glmnet(x,y,alpha=1)

plot(fit.lasso,xvar="lambda",label=TRUE)

cv.lasso=cv.glmnet(x,y)

coef(cv.lasso)

Suppose after this command, we get 4 variables which have non-zero coefficient value, i.e: x1, x2, x3, x4.

Then, I used this command:

fit = lm( y~x1+x2+x3+x4)

And check p values corresponding to x1, x2, x3, x4. Then if x3 have p-value > 0.05, do I need to remove it in the final model?

Or , using Coef(CV.lasso) we get coefficients values use for predicting y?

And hints?

I am using R

Answer 1

After you have done LASSO you should generally NOT use the selected variables in a separate linear regression.

There are several ways to select a subset of predictor variables for a model. For example, you could use stepwise regression or, with few enough predictors, you could examine all possible subsets of predictors. In these cases a criterion like AIC is used to trade off the fit against the number of variables included.

But in common use the selected variables are then simply incorporated into a standard linear regression. The p-values in that linear regression are not valid, as they do not incorporate the fact that you had already performed outcome-based variable selection. Also, if there are correlations among predictors, the particular variables that you choose can depend heavily upon the particular data sample you analyzed. Making this worse, the regression coefficients for those selected from a set of correlated predictors will tend to be larger in magnitude than their true values in the population. Thus the results from these types of linear models can have poor performance on new samples from the population. You can examine these behaviors by analyzing multiple bootstrap samples from your data set.

Although LASSO also may select different sets of variables on different data samples, it has a major advantage over those other approaches. It also penalizes the regression coefficients of the selected variables, lowering their magnitudes from those in a standard linear regression. This penalization typically improves the ability to predict results on new data samples.

So if you simply take the LASSO-selected variables and put them into a new linear regression, not only do you have the problems imposed by all variable selection approaches but also you have lost the LASSO advantage of penalizing coefficients of the selected variables to improve prediction. The p-values for that linear regression will be no more valid than for stepwise or best-subset selection. (Removing "insignificant" predictors in a multiple regression based on a p-value cutoff is not a good idea in any event).

So your proposed approach would undo the good that you did by choosing a principled approach like LASSO, with its penalization, in the first place. As Richard Hardy notes in a comment, there can be ways to use LASSO for variable selection to incorporate into linear regressions, but those are specialized multi-step approaches for particular circumstances, and they don't seem to give any advantage over glmnet() in your application.

So stick with the predictors and coefficients that LASSO provided.