Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
These are the linear regression assumptions:
Linearity: The relationship between X and the mean of Y is linear. Homoscedasticity: The variance of residual is the same for any value of X. Independence: Observations are independent of each other. Normality: For any fixed value of X, Y is normally distributed.
Do they apply to Lasso Regression, too?
I will be a contrarian and say that most assumptions do not apply to LASSO regression.
In the classical linear model, those assumptions are used to show that the OLS estimator is the minimum-variance linear unbiased estimator (Gauss-Markov theorem) and to have correct t-stats, F-stats, and confidence intervals.
In LASSO regression, those are less important. LASSO gives a biased estimator, so any of the Gauss-Markov business for the minimum-variance linear unbiased estimator no longer applies; indeed, LASSO is not even a linear estimator of the coefficients, so Gauss-Markov doubly does not apply. Then the hypothesis testing and confidence intervals would not be of primary concern for a LASSO regression. Indeed, it does not even seem agreed upon how such inference would be performed.
(Linearity still matters, but that’s because OLS and LASSO both estimate the same parameters, which are the coefficients on the features.)
Overall, the typical assumptions are assumed in an OLS linear regression because of nice properties of later inferences that are not particularly important for a situation where LASSO regression would be applied.
Normality is not an assumption of linear regression.
Yes, they do. Lasso regression is a linear regression with a penalty term on the magnitude of the coefficients; the penalty term in no way affects the structure of the underlying model (linearity, independence, homoskedasticity) and the assumptions are the same.
Yes, they are valid also for Lasso, Ridge Regression and Elastic Net. I think you are referring to classical linear model (CLM) assumptions for cross-sectional regression:
where the first four are used to establish unbiasedness of OLS, whereas the fifth is employed to derive the usual variance formulas and to conclude that OLS is Best linear unbiased.
One can also assume normality of the error term to obtain the exact sampling distribution of t statistics and F statistics, so that one can carry out exact hypotheses tests, but normality is not a necessary assumption for CLM.
