1
$\begingroup$

What are the assumptions of Ridge and LASSO Regression? Which assumptions of Linear Regression can be done away with in Ridge and LASSO Regressions?

$\endgroup$
1
$\begingroup$

This is a broad question and I am sure a search on this site would return several results. Nevertheless here is a couple of things to remember.

The basic thing to remember about Ridge and Lasso is that they are both parametric methods. What this means is that for them to be applicable, a specific model has to be postulated, usually a linear one:

$$\mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}$$

The major advantage of these methods compared to OLS is that they can handle multicollinearity, i.e. a predictor matrix with rank less than the number of its columns.

Another thing to remember is that neither Ridge nor Lasso actually respond well to outlying observations. This may be seen most easily for the case of an orthonormal predictor matrix as then the estimators may be written as (unbounded) functions of the notoriously non-robust OLS estimator. Therefore, much like the OLS estimator, Ridge and Lasso should be used with caution in non-clean datasets.

$\endgroup$
  • 1
    $\begingroup$ What about the assumptions of homoscedasticity and normality? Could I drop them? $\endgroup$ – Shreyo Mallik Mar 14 '17 at 23:51
  • 1
    $\begingroup$ @ShreyoMallik Normality is not necessary for OLS regression, the Gauss-Markov theorem has no need for it. Homoscedasticity is a more persistent problem, however. $\endgroup$ – JohnK Mar 14 '17 at 23:53
  • $\begingroup$ Then, why do we assume normality and homoscedasticity in OLS Regression? Could you please elucidate? $\endgroup$ – Shreyo Mallik Mar 15 '17 at 0:08
  • $\begingroup$ @ShreyoMallik You assume normality for inference purposes, e.g. t-tests, and homoscedasticity to prove that the OLS is BLUE, i.e. the Gauss-Markov theorem. Neither of these assumptions are crucial in OLS regression and there are work-arounds if they are violated. $\endgroup$ – JohnK Mar 15 '17 at 0:12
  • 1
    $\begingroup$ @ShreyoMallik, as JohnK correctly said, normality is not assumed for OLS (assuming normality is a very common mistake). Its only use is in small sample inference. In large sample inference it is not needed anymore due to the central limit theorem. Also, consistency, asymptotic normality and Gauss-Markov theorem does not rest on the assumption of normality. $\endgroup$ – Richard Hardy Mar 15 '17 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.