# Interpreting the biased coefficients of a ridge or LASSO regression model [duplicate]

In a recent conversation with one of the colleagues I was presented with a view that LASSO/Ridge regularization (trading bias for variance) renders coefficient estimates useless for interpretation, i.e. LASSO/Ridge coefficients can't be interpreted in the same way as OLS coefficients due to increase in bias. While I don't agree with this, I'd like to hear some thoughts on this to prepare a more coherent counterargument.

• The edited title strikes me as better reflecting the question, since the bias-variance decomposition is rather unrelated, though it is easy to edit it back if you do not like it.
– Dave
Mar 23 at 0:38
• What does this colleague mean by "interpreting" OLS coefficients? Mar 23 at 10:09
• Interpreting in the standard fashion: a unit change in X leads to a change in Y equal to coefficient for X. I guess the mindset here is that due to introduction of bias coefficients somehow lose "physical" meaning associated with underlying real world processes. Mar 23 at 10:20
• Not at all! The interpretation of a coefficient does not depend on how that coefficient is estimated -- the coefficient still means the same thing it always did.
– whuber
Mar 23 at 13:31

1. Many common estimators are biased. Ever calculated $$S = \sqrt{\dfrac{\overset{n}{\underset{i=1}{\sum}}\left(X_i-\bar X\right)^2}{n-1}}$$ as the standard deviation? That is a biased estimator for the standard deviation. Ever fit a logistic regression? You used a biased estimator of the true coefficients. Despite this, however, you probably felt comfortable interpreting those statistics (as is reasonable). Consider asking your colleague about these. (I concede that the Workplace Stack Exchange might advise otherwise.) If your colleague protests, "But they are consistent," so are ridge and LASSO. (Yes, that requires some assumptions, but OLS unbiasedness requires assumptions, too.)
2. This is subtle, but I dispute the idea that you are interpreting the estimator. You interpret the model parameters, and you use estimators to guess what those unknown parameters are. The ridge and LASSO estimators are just estimating the $$\beta$$ parameter vector of $$\mathbb E\left[Y\vert X\right] = X\beta$$, same as OLS. It might be that you are in a situation where ridge or LASSO has inferior properties compared to the OLS estimator (maybe a reviewer insists on an unbiased estimator), but all of these are just ways of guessing the true values of the parameters.