13
$\begingroup$

Examples of this page show that simple regression is markedly affected by outliers and this can be overcome by techniques of robust regression: http://www.alastairsanderson.com/R/tutorials/robust-regression-in-R/ . I believe lmrob and ltsReg are other robust regression techniques.

Why should one not do robust regression (like rlm or rq) every time rather than performing simple regression (lm)? Are there any drawbacks of these robust regression techniques? Thanks for your insight.

$\endgroup$
1
  • 2
    $\begingroup$ This might be helpful. $\endgroup$ Apr 13, 2015 at 8:22

1 Answer 1

7
$\begingroup$

The Gauss-Markov theorem:

In a linear model with spherical errors (which along the way includes an assumption of no outliers, via a finite error variance), OLS is efficient in a class of linear unbiased estimators - there are (restrictive, to be sure) conditions under which "you can't do better than OLS".

$\endgroup$
4
  • $\begingroup$ So if there are no outliers,linear regression would be the best. But if there are, or if other assumptions are being violated, then only one should perform robust regressions. Is that correct? $\endgroup$
    – rnso
    Apr 13, 2015 at 12:29
  • 2
    $\begingroup$ If there are outliers, other techniques are better, yes. I would not jump to the conclusion that "if other assumptions are being violated, then [...] one should perform robust regressions" - it is not a cure-all for all violations. F.x., when errors are correlated with regressors and you are after causal effects, instrumental variables techniques are called for. $\endgroup$ Apr 13, 2015 at 13:52
  • $\begingroup$ The Gauss-Markov theorem does not make a normality assumption. We still get the BLUE if the error is heavy-tailed. $\endgroup$
    – Dave
    Oct 14, 2020 at 14:21
  • $\begingroup$ Correct about normality; nor do I say it does. It does make an assumption of finite error variances, though, so does that square with "heavy tails" to you? $\endgroup$ Oct 14, 2020 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.