LASSO assumptions

Question

In a LASSO regression scenario where

$y= X \beta + \epsilon$,

and the LASSO estimates are given by the following optimization problem

$ \min_\beta ||y - X \beta|| + \tau||\beta||_1$

Are there any distributional assumptions regarding the $\epsilon$?

In an OLS scenario, one would expect that the $\epsilon$ are independent and normally distributed.

Does it make any sense to analyze the residuals in a LASSO regression?

I know that the LASSO estimate can be obtained as the posterior mode under independent double-exponential priors for the $\beta_j$. But I haven't found any standard "assumption checking phase".

Thanks in advance (:

Answer 1

I am not an expert on LASSO, but here is my take.

First note that OLS is pretty robust to violations of indepence and normality. Then judging from the Theorem 7 and the discussion above it in the article Robust Regression and Lasso (by X. Huan, C. Caramanis and S. Mannor) I guess, that in LASSO regression we are more concerned not with the distribution of $\varepsilon_i$, but in the joint distribution of $(y_i,x_i)$. The theorem relies on the assumption that $(y_i,x_i)$ is a sample, so this is comparable to usual OLS assumptions. But LASSO is less restrictive, it does not constrain $y_i$ to be generated from the linear model.

To sum up, the answer to your first question is no. There are no distributional assumptions on $\varepsilon$, all distributional assumptions are on $(y,X)$. Furthermore they are weaker, since in LASSO nothing is postulate on conditional distribution $(y|X)$.

Having said that, the answer to the second question is then also no. Since the $\varepsilon$ does not play any role it does not make any sense to analyse them the way you analyse them in OLS (normality tests, heteroscedasticity, Durbin-Watson, etc). You should however analyse them in context how good the model fit was.