1
$\begingroup$

What are the assumptions of Ridge and LASSO Regression? Which assumptions of Linear Regression can be done away with in Ridge and LASSO Regressions?

$\endgroup$
4

1 Answer 1

2
$\begingroup$

This is a broad question and I am sure a search on this site would return several results. Nevertheless here is a couple of things to remember.

The basic thing to remember about Ridge and Lasso is that they are both parametric methods. What this means is that for them to be applicable, a specific model has to be postulated, usually a linear one:

$$\mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}$$

The major advantage of these methods compared to OLS is that they can handle multicollinearity, i.e. a predictor matrix with rank less than the number of its columns.

Another thing to remember is that neither Ridge nor Lasso actually respond well to outlying observations. This may be seen most easily for the case of an orthonormal predictor matrix as then the estimators may be written as (unbounded) functions of the notoriously non-robust OLS estimator. Therefore, much like the OLS estimator, Ridge and Lasso should be used with caution in non-clean datasets.

$\endgroup$
11
  • 1
    $\begingroup$ What about the assumptions of homoscedasticity and normality? Could I drop them? $\endgroup$ Mar 14, 2017 at 23:51
  • 1
    $\begingroup$ @ShreyoMallik Normality is not necessary for OLS regression, the Gauss-Markov theorem has no need for it. Homoscedasticity is a more persistent problem, however. $\endgroup$
    – JohnK
    Mar 14, 2017 at 23:53
  • $\begingroup$ Then, why do we assume normality and homoscedasticity in OLS Regression? Could you please elucidate? $\endgroup$ Mar 15, 2017 at 0:08
  • $\begingroup$ @ShreyoMallik You assume normality for inference purposes, e.g. t-tests, and homoscedasticity to prove that the OLS is BLUE, i.e. the Gauss-Markov theorem. Neither of these assumptions are crucial in OLS regression and there are work-arounds if they are violated. $\endgroup$
    – JohnK
    Mar 15, 2017 at 0:12
  • 1
    $\begingroup$ @ShreyoMallik, as JohnK correctly said, normality is not assumed for OLS (assuming normality is a very common mistake). Its only use is in small sample inference. In large sample inference it is not needed anymore due to the central limit theorem. Also, consistency, asymptotic normality and Gauss-Markov theorem does not rest on the assumption of normality. $\endgroup$ Mar 15, 2017 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.