Goodness of fit test for LASSO

Question

How would you do a goodness of fit test for Lasso regression?

Im guessing that the $R^2$ value, as for linear regression, wont work anymore. Why is that?

Answer 1

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!