16
$\begingroup$

In a LASSO regression scenario where

$y= X \beta + \epsilon$,

and the LASSO estimates are given by the following optimization problem

$ \min_\beta ||y - X \beta|| + \tau||\beta||_1$

Are there any distributional assumptions regarding the $\epsilon$?

In an OLS scenario, one would expect that the $\epsilon$ are independent and normally distributed.

Does it make any sense to analyze the residuals in a LASSO regression?

I know that the LASSO estimate can be obtained as the posterior mode under independent double-exponential priors for the $\beta_j$. But I haven't found any standard "assumption checking phase".

Thanks in advance (:

$\endgroup$
14
$\begingroup$

I am not an expert on LASSO, but here is my take.

First note that OLS is pretty robust to violations of indepence and normality. Then judging from the Theorem 7 and the discussion above it in the article Robust Regression and Lasso (by X. Huan, C. Caramanis and S. Mannor) I guess, that in LASSO regression we are more concerned not with the distribution of $\varepsilon_i$, but in the joint distribution of $(y_i,x_i)$. The theorem relies on the assumption that $(y_i,x_i)$ is a sample, so this is comparable to usual OLS assumptions. But LASSO is less restrictive, it does not constrain $y_i$ to be generated from the linear model.

To sum up, the answer to your first question is no. There are no distributional assumptions on $\varepsilon$, all distributional assumptions are on $(y,X)$. Furthermore they are weaker, since in LASSO nothing is postulate on conditional distribution $(y|X)$.

Having said that, the answer to the second question is then also no. Since the $\varepsilon$ does not play any role it does not make any sense to analyse them the way you analyse them in OLS (normality tests, heteroscedasticity, Durbin-Watson, etc). You should however analyse them in context how good the model fit was.

$\endgroup$
  • $\begingroup$ The link is broken. $\endgroup$ – Galled Oct 18 '12 at 0:13
  • $\begingroup$ Thanks, fixed it. Also added more information about the article in case the link is broken again. $\endgroup$ – mpiktas Oct 18 '12 at 7:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.