Skip to main content
added 16 characters in body
Source Link
goangit
  • 566
  • 3
  • 12

Your question is not entirely clear. This answer assumes that by "distribution of the features" you mean the conditional distributions of the response across explanatory variables.

Under the General Linear Model regression estimates are obtained by minimising the Residual Sum of Squares

$RSS = \sum\limits_{i=1}^{n} \Big(y_i - \beta_0 - \sum\limits_{j=1}^{p} \beta_j x_{ij} \Big)^2$.

In contrast, Ridge Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p \beta_j^2$,

and Lasso Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p |\beta_j|$,

where $\lambda$ is a tuning parameter.

So, Ridge Regression and Lasso Regression are special cases of the General Linear Model. They add penalty terms but otherwise all of the same conditions apply, including conditionally independent Gaussian residuals with zero mean and constant variance across the range of the explanatory variable(s).

Your question is not entirely clear. This answer assumes that by "distribution of the features" you mean the conditional distributions of the explanatory variables.

Under the General Linear Model regression estimates are obtained by minimising the Residual Sum of Squares

$RSS = \sum\limits_{i=1}^{n} \Big(y_i - \beta_0 - \sum\limits_{j=1}^{p} \beta_j x_{ij} \Big)^2$.

In contrast, Ridge Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p \beta_j^2$,

and Lasso Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p |\beta_j|$,

where $\lambda$ is a tuning parameter.

So, Ridge Regression and Lasso Regression are special cases of the General Linear Model. They add penalty terms but otherwise all of the same conditions apply, including conditionally independent Gaussian residuals with zero mean and constant variance across the range of the explanatory variable(s).

Your question is not entirely clear. This answer assumes that by "distribution of the features" you mean the conditional distributions of the response across explanatory variables.

Under the General Linear Model regression estimates are obtained by minimising the Residual Sum of Squares

$RSS = \sum\limits_{i=1}^{n} \Big(y_i - \beta_0 - \sum\limits_{j=1}^{p} \beta_j x_{ij} \Big)^2$.

In contrast, Ridge Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p \beta_j^2$,

and Lasso Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p |\beta_j|$,

where $\lambda$ is a tuning parameter.

So, Ridge Regression and Lasso Regression are special cases of the General Linear Model. They add penalty terms but otherwise all of the same conditions apply, including conditionally independent Gaussian residuals with zero mean and constant variance across the range of the explanatory variable(s).

Source Link
goangit
  • 566
  • 3
  • 12

Your question is not entirely clear. This answer assumes that by "distribution of the features" you mean the conditional distributions of the explanatory variables.

Under the General Linear Model regression estimates are obtained by minimising the Residual Sum of Squares

$RSS = \sum\limits_{i=1}^{n} \Big(y_i - \beta_0 - \sum\limits_{j=1}^{p} \beta_j x_{ij} \Big)^2$.

In contrast, Ridge Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p \beta_j^2$,

and Lasso Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p |\beta_j|$,

where $\lambda$ is a tuning parameter.

So, Ridge Regression and Lasso Regression are special cases of the General Linear Model. They add penalty terms but otherwise all of the same conditions apply, including conditionally independent Gaussian residuals with zero mean and constant variance across the range of the explanatory variable(s).