13
$\begingroup$

Examples of this page show that simple regression is markedly affected by outliers and this can be overcome by techniques of robust regression: http://www.alastairsanderson.com/R/tutorials/robust-regression-in-R/ . I believe lmrob and ltsReg are other robust regression techniques.

Why should one not do robust regression (like rlm or rq) every time rather than performing simple regression (lm)? Are there any drawbacks of these robust regression techniques? Thanks for your insight.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ This might be helpful. $\endgroup$
    – conjugateprior
    Commented Apr 13, 2015 at 8:22

1 Answer 1

Reset to default
7
$\begingroup$

The Gauss-Markov theorem:

In a linear model with spherical errors (which along the way includes an assumption of no outliers, via a finite error variance), OLS is efficient in a class of linear unbiased estimators - there are (restrictive, to be sure) conditions under which "you can't do better than OLS".

Improve this answer
$\endgroup$
10
  • $\begingroup$ So if there are no outliers,linear regression would be the best. But if there are, or if other assumptions are being violated, then only one should perform robust regressions. Is that correct? $\endgroup$
    – rnso
    Commented Apr 13, 2015 at 12:29
  • 2
    $\begingroup$ If there are outliers, other techniques are better, yes. I would not jump to the conclusion that "if other assumptions are being violated, then [...] one should perform robust regressions" - it is not a cure-all for all violations. F.x., when errors are correlated with regressors and you are after causal effects, instrumental variables techniques are called for. $\endgroup$
    – Christoph Hanck
    Commented Apr 13, 2015 at 13:52
  • $\begingroup$ The Gauss-Markov theorem does not make a normality assumption. We still get the BLUE if the error is heavy-tailed. $\endgroup$
    – Dave
    Commented Oct 14, 2020 at 14:21
  • $\begingroup$ Correct about normality; nor do I say it does. It does make an assumption of finite error variances, though, so does that square with "heavy tails" to you? $\endgroup$
    – Christoph Hanck
    Commented Oct 14, 2020 at 15:10
  • $\begingroup$ However Gauss-Markov considers only linear estimators. Robust estimators are nonlinear. $\endgroup$
    – dave fournier
    Commented Dec 4, 2023 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.