1
$\begingroup$

Including too few variables in OLS regression means that the coefficient estimates can be biased, e.g. if we aren't controlling for a variable in a model that should be there, it is instead captured by the error term, so an estimated coefficient could be overestimating the effect of an independent variable on the dependent variable.

When it comes to ordered logistic regression (dependent variable has an ordinal ranking of say three categories), does the effect of an omitted independent variable follow the same logic as in OLS, where the estimated coefficients can be biased?

Help appreciated!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

Yes, the same holds true.

Note that omitted variable bias is an epistemic, not a technical problem. It means you lack a meaningful interpretation of results from a statistical model, e.g. coefficients, because the selection of variables you allowed the model to use was misleading. It has nothing to do with the technical side of your model, e.g., which estimator you use, which link functions you use (logit, in your case), which fitting algorithm etc.

Coefficients will almost always change in some way when you add or remove variables from your model. Whether this is a bias really depends on the question you are out to answer and your expert knowledge on the topic. This links to the broader problem of confounding and the assumptions you need to make to interpret your result from a causal perspective. If that's your aim, leaving out important confounding variables from your model can totally mess up your conclusions, as in typical 'spurious associations': https://www.tylervigen.com/spurious-correlations

But note that you can also use too many variables, from an espistemic perspective (besides the technical and estimation issues that arise in crowded models), for example when conditioning on a collider!

A good introduction into these problems is chapter 5+6 in:

  • McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan. CRC press.

Besides such theoretical approaches for the inclusion of variables into models, there are also data-driven approaches, which can be useful depending on what your goal is ([causal / statistical] inference? prediction?), a really good primer is:

  • Heinze, G., & Dunkler, D. (2017). Five myths about variable selection. Transplant International, 30(1), 6-10.
Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Nice answer. I want to add that with nonlinear models (e.g., logit), coefficients can change value and meaning depending on the other variables in the model even without confounding. This is a third problem of collapsibility, which is worth remembering as well. $\endgroup$
    – Noah
    Jun 5, 2020 at 20:38
  • $\begingroup$ Following up on the comment from @Noah I'd suggest a look at this page. It shows how, in applications like logistic regression that go beyond OLS, omitting a predictor associated with outcome can lead to bias in estimating coefficients for the other predictors even if it isn't correlated with them. So the problems with omitted variables are even worse in logistic regression than in OLS, and the tradeoffs between omitting variables and overfitting are more daunting. $\endgroup$
    – EdM
    Jun 5, 2020 at 21:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.