# MSE of an individual coefficient from ridge or lasso vs. OLS

Consider a multiple regression model

$$y = X\beta + \varepsilon$$

with $K$ regressors in $X$.

If the model is correctly specified, the OLS estimator $\hat\beta_{OLS}=(X'X)^{-1}X'y$ will be the minimum-variance (or best) linear unbiased estimator (MVLUE or BLUE) of $\beta$.

When it comes to minimizing the mean squared error (MSE), ridge regression and lasso may provide estimators $\hat\beta_{ridge}(\lambda_{ridge})$ and $\hat\beta_{lasso}(\lambda_{lasso})$, respectively, with smaller MSEs than that of OLS for some intervals of penalty intensity $\lambda_{ridge}$ and $\lambda_{lasso}$.

As far as I understand, this concerns the entire parameter vector $\beta$ rather than each individual parameter $\beta_k$ for $k=1,\dotsc,K$.

Question: Can something concrete and useful be said about the individual parameter estimates $\hat\beta_{k,OLS}$ versus $\hat\beta_{k,ridge}$ and $\hat\beta_{k,lasso}$ in terms of MSE?

• Nitpick: it's not minimizing MSE "instead of [minimizing] variance", it's minimizing MSE instead of minimizing variance and guaranteeing unbiasedness. – amoeba Aug 1 '16 at 16:29
• @amoeba, good point. I have now omitted the whole "instead" clause. – Richard Hardy Aug 1 '16 at 17:35