What are distribution assumptions in Ridge and Lasso regression models?

Question

What are the assumptions for the distribution of the features for regression models like Lasso regression or Ridge regression? Why is it better to have features with Gaussian distributions?

Answer 1

Your question is not entirely clear. This answer assumes that by "distribution of the features" you mean the conditional distributions of the response across explanatory variables.

Under the General Linear Model regression estimates are obtained by minimising the Residual Sum of Squares

$RSS = \sum\limits_{i=1}^{n} \Big(y_i - \beta_0 - \sum\limits_{j=1}^{p} \beta_j x_{ij} \Big)^2$.

In contrast, Ridge Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p \beta_j^2$,

and Lasso Regression aims to minimise

$RSS + \lambda \; \sum\limits_{j=1}^p |\beta_j|$,

where $\lambda$ is a tuning parameter.

So, Ridge Regression and Lasso Regression are special cases of the General Linear Model. They add penalty terms but otherwise all of the same conditions apply, including conditionally independent Gaussian residuals with zero mean and constant variance across the range of the explanatory variable(s).

Answer 2

From a Bayesian standpoint, the assumptions are simply in the priors on the coefficients. Ridge regression is equivalent to using a Gaussian prior, whereas LASSO is equivalent to using a Laplace prior. As @whuber said, these models don't make assumptions on the distribution of the explanatory variables.