Skip to main content
added 1 character in body
Source Link
Dave
  • 67.2k
  • 7
  • 105
  • 305

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via \hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$$\hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via \hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!

Source Link
Dave
  • 67.2k
  • 7
  • 105
  • 305

The LASSO regression is still a linear regression.

In vanilla linear regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{ols} = (X^TX)^{-1}X^Ty$.

In ridge (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via \hat{\beta}_{ridge,\lambda} = (X^TX + \lambda I)^{-1} X^Ty$.

In LASSO (linear) regression, we posit that the phenomenon follows the following:

$$y = X\beta + \epsilon$$

And we estimate $\beta$ via $\hat{\beta}_{LASSO,\lambda} = \text{no closed-form expression}$.

Well we can argue forever about which estimator we want to use. Even for variance, it is not universal that we should be dividing by $n-1$ instead of $n$. However, we use all three estimators (OLS, ridge, and LASSO) to guess the $\beta$ in $y = X\beta + \epsilon$.

$R^2$ might not be ideal for LASSO regression, but it might not be ideal for OLS regression, either!