2
$\begingroup$

How would you do a goodness of fit test for Lasso regression?

Im guessing that the $R^2$ value, as for linear regression, wont work anymore. Why is that?

$\endgroup$
4
  • $\begingroup$ Why do you guess that? $\endgroup$
    – Dave
    Sep 11 '20 at 14:39
  • $\begingroup$ just a feeling, i dont have anymore justificaiton than that (which is worrying) @Dave $\endgroup$
    – Trajan
    Sep 11 '20 at 14:40
  • $\begingroup$ And what do you mean by "goodness of fit"? Would mean squared error suffice for your purposes? $\endgroup$
    – Dave
    Sep 11 '20 at 14:42
  • $\begingroup$ possibly, the first sentence i saw as an interview question, so i guess so @Dave $\endgroup$
    – Trajan
    Sep 11 '20 at 14:43
1
$\begingroup$

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!

$\endgroup$
2
  • $\begingroup$ i thought for OLS regression, it is least squares which is the estimator? $\endgroup$
    – Trajan
    Sep 11 '20 at 16:05
  • $\begingroup$ Squared error is the type of error that we aim to minimize, and it is the case that the $\hat{\beta}_{ols}$ that I gave minimizes $\sum(y_i - \hat{y}_i)^2$, hence the term "least squares" for "least squared error". $\endgroup$
    – Dave
    Sep 11 '20 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.