Goodness of fit test for LASSO

How would you do a goodness of fit test for Lasso regression?

Im guessing that the $$R^2$$ value, as for linear regression, wont work anymore. Why is that?

• Why do you guess that? – Dave Sep 11 '20 at 14:39
• just a feeling, i dont have anymore justificaiton than that (which is worrying) @Dave – Trajan Sep 11 '20 at 14:40
• And what do you mean by "goodness of fit"? Would mean squared error suffice for your purposes? – Dave Sep 11 '20 at 14:42
• possibly, the first sentence i saw as an interview question, so i guess so @Dave – Trajan Sep 11 '20 at 14:43

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $$\beta$$ via $$\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $$\beta$$ via $$\hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $$\beta$$ via $$\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $$n-1$$ instead of $$n$$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $$\beta$$ in $$y = X\beta + \epsilon$$.

$$R^2$$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!

• i thought for OLS regression, it is least squares which is the estimator? – Trajan Sep 11 '20 at 16:05
• Squared error is the type of error that we aim to minimize, and it is the case that the $\hat{\beta}_{ols}$ that I gave minimizes $\sum(y_i - \hat{y}_i)^2$, hence the term "least squares" for "least squared error". – Dave Sep 11 '20 at 16:09