3
$\begingroup$

I find following commonly mentioned linear regression methods:

OLS: ordinary least squares

GLS: generalized least squares

WLS: weighted least squaes

RLM: robust linear model

OLS is usually the default. I believe robust model is to be used to correctly handle outliers, but I am not clear about others.

What are the criteria to choose one over the other?

Edit: It is mentioned in the comments that it is a very broad question (I did not know that!). However, I would like to have a one or two lines on each of above to know the "indications" or when to use them.

OLS: default

RLM: if outliers are important and cannot be ignored.

That leaves only GLS and WLS. What would be most important reasons to use them?

$\endgroup$
3
  • $\begingroup$ This is very general, and whole books are needed for an answer. Could you limit the scope somewhat? Linear regression methods for ... ? $\endgroup$ Jul 12, 2020 at 17:35
  • $\begingroup$ I have tried to specify that I need only a general indication on when to use each of these. $\endgroup$
    – rnso
    Jul 12, 2020 at 17:40
  • 2
    $\begingroup$ OLS is useful as pedagogically to generally introduce regression and it's concepts and assumptions, but I am not sure if or when I actually have used OLS in research. Regression methods permitting creative violation of OLS regression's assumptions permit us to model the complexities with which the world behaves with more fidelity. $\endgroup$
    – Alexis
    Jul 12, 2020 at 17:49

1 Answer 1

3
$\begingroup$

Your links goes to statsmodels program web pages, a software I do not know. I will assume their use of terms is the standard. A very general indication, just as a starter, what you really need is a book on regression.

  • OLS is the starting point, many other models can be seen as extensions or generalization. Assumptions is continuous response, linear effects and constant variance + independence (of residuals.)

  • GLS weakens assumption, do not assume constant variance nor independence. So you will need somehow to model the variance and covariances.

  • WLS is GLS but with covariances zero, so really an assumption of independence of residuals.

  • RLM is really a huge class of models and methods. Especially think about this for routine or automatized analyses.

$\endgroup$
4
  • $\begingroup$ I had asked earlier stats.stackexchange.com/questions/146077/… . But I cannot understand how to adjust options in RLM since you say that RLM is "a huge class and methods". $\endgroup$
    – rnso
    Jul 14, 2020 at 3:11
  • $\begingroup$ It is suggested at stats.stackexchange.com/questions/473603/… that WLS is useful if there is a problem of Heteroscedasticity. $\endgroup$
    – rnso
    Jul 15, 2020 at 0:55
  • $\begingroup$ Well, that is what I tried to say, do not assume constant variance. $\endgroup$ Jul 15, 2020 at 1:21
  • $\begingroup$ as far as i know, the only difference between OLS and WLS is the weights that get multiplied as part of whitening, while no whitening is applicable for OLS. given this, i would like to know, how can assumptions become less stringent in WLS from OLS and how WLS can be useful when there is heteroscedasticity? $\endgroup$ Mar 28, 2022 at 1:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.