I find following commonly mentioned linear regression methods:

OLS: ordinary least squares

GLS: generalized least squares

WLS: weighted least squaes

RLM: robust linear model

OLS is usually the default. I believe robust model is to be used to correctly handle outliers, but I am not clear about others.

What are the criteria to choose one over the other?

Edit: It is mentioned in the comments that it is a very broad question (I did not know that!). However, I would like to have a one or two lines on each of above to know the "indications" or when to use them.

OLS: default

RLM: if outliers are important and cannot be ignored.

That leaves only GLS and WLS. What would be most important reasons to use them?

  • $\begingroup$ This is very general, and whole books are needed for an answer. Could you limit the scope somewhat? Linear regression methods for ... ? $\endgroup$ – kjetil b halvorsen Jul 12 '20 at 17:35
  • $\begingroup$ I have tried to specify that I need only a general indication on when to use each of these. $\endgroup$ – rnso Jul 12 '20 at 17:40
  • 2
    $\begingroup$ OLS is useful as pedagogically to generally introduce regression and it's concepts and assumptions, but I am not sure if or when I actually have used OLS in research. Regression methods permitting creative violation of OLS regression's assumptions permit us to model the complexities with which the world behaves with more fidelity. $\endgroup$ – Alexis Jul 12 '20 at 17:49

Your links goes to statsmodels program web pages, a software I do not know. I will assume their use of terms is the standard. A very general indication, just as a starter, what you really need is a book on regression.

  • OLS is the starting point, many other models can be seen as extensions or generalization. Assumptions is continuous response, linear effects and constant variance + independence (of residuals.)

  • GLS weakens assumption, do not assume constant variance nor independence. So you will need somehow to model the variance and covariances.

  • WLS is GLS but with covariances zero, so really an assumption of independence of residuals.

  • RLM is really a huge class of models and methods. Especially think about this for routine or automatized analyses.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.