Can I use a linear model estimated with lasso for intepretation?

Question

If all the assumptions are correct, a linear model can be used for interpretation. It is possible to understand if a variable has a significant effect on the response and if so, it is also possible to understand which is the mean variation of the response when a predictor varies from x to x+1 (while the other predictors remain fixed).
I don't understand if I can use a linear model fitted with lasso for interpretation or not.
Can I say that predictors whose coefficients are set to zero by the lasso have not a significant effect on the response? Can I say that the coefficient represents the mean variation of the response when a predictor varies from x to x+1 (while the other predictors remain fixed)? Do I have to do diagnostics for the model estimated with lasso too to understand if the assumptions are correct? Thank you.

Answer 1

First, in terms of interpretation of linear models, I'm assuming that when you say "It is possible to understand if a variable has a significant effect on the response" you mean this in the sense of an association with the response rather than a causal effect on the response. Unless you have other evidence of a causal relation, it's important to guard against the temptation of causal inference from a regression.

In terms of the model that you chose via LASSO, the interpretation of the regression coefficients is as in linear regression, insofar as they represent the relation of the expected change in the response variable with a change in an independent variable, when all the other independent variables in your model are taken into account.

But the variable-selection process in LASSO necessarily adds some caution to your interpretation. Most important, variables omitted should not be considered unrelated to the response. It's just that, based on your criterion for choosing the $L_1$ penalty, they weren't as important as the included variables on the particular data set you analyzed. This is a particular problem when the variable selection was from a set of highly correlated independent variables; the choice by LASSO from data sample to data sample could be highly variable. You could explore this possibility in your own data by repeating the LASSO on multiple bootstrap samples and seeing whether you always select the same independent variables.

So to the interpretation in the second paragraph of my answer, perhaps one should add "and with any additional contributions of the omitted variables being ignored."

In terms of diagnostics, they don't call this site "Cross Validated" for nothing. If you haven't already, get a copy of An Introduction to Statistical Learning and learn from it.

Answer 2

Give a set of input measurements x1, x2 ...xp and an outcome measurement y, the lasso fits a linear model

yhat=b0 + b1*x1+ b2*x2 + ... bp*xp

The criterion it uses is:

Minimize sum( (y-yhat)^2 ) subject to sum[absolute value(bj)] <= s

The first sum is taken over observations (cases) in the dataset. The bound "s" is a tuning parameter. When "s" is large enough, the constraint has no effect and the solution is just the usual multiple linear least squares regression of y on x1, x2, ...xp.

However when for smaller values of s (s>=0) the solutions are shrunken versions of the least squares estimates. Often, some of the coefficients bj are zero. Choosing "s" is like choosing the number of predictors to use in a regression model, and cross-validation is a good tool for estimating the best value for "s".

This means, that when you found correct "s", you in essence had minimized the risk, that zeroed coefficients affected response too much. So, just use this instrument carefully, and you are safe!