10
$\begingroup$

For a last couple of weeks I've been thinking about OVB (Omitted variable bias) in the context of regression and solution for that (how to avoid this problem). I am acquainted with Shalizi's lectures (2.2), but he is just describing this mathematically.

This week somebody said that it's quite easy - the solution for OVB is to include all those predictors that control the effect of confounding covariates, not all predictors for dependent variable Y.

I am not sure if this is true and yes, I do feel that I lack of deeper knowledge.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Can you link to these lectures? $\endgroup$
    – Matthew Drury
    Nov 9, 2017 at 19:49

4 Answers 4

Reset to default
8
$\begingroup$

This is not necessarily wrong, but not always feasible and also not a free lunch.

An omitted variable may cause (see, e.g., the comments below for additional thoughts on the matter) bias if it is both (a) related to the outcome $Y$ and (b) correlated with the predictor $X$ whose effect on $Y$ you are primarily interested in.

Consider an example: You want to learn about the causal effect of additional schooling on later earnings. Another variable that is most certainly satisfies the conditions (a) and (b) is "motivation" - more motivated people will both be more successful in their jobs (whether they are highly schooled or not) and generally choose to receive more education, as they are likely to like learning, and not find it too painful to study for exams.

So, when comparing earnings of highly schooled and less schooled employees without controlling for motivation, you would likely at least partially not be comparing two groups that only differ in terms of their schooling (whose effect you are interested in) but also in terms of their motivation, so the observed difference in earnings should not only be ascribed to differences in schooling.

Now, it would indeed be a solution to control for motivation by including it into the regression. The likely problem is of course: are you going to have data on motivation? Even if you were to conduct a survey yourself (rather than use say administrative data, that will most likely not have entries on motivation), how would you even measure it?

As to why including everything is not a free lunch: if you have a small sample, including all available covariates may quickly lead to overfitting when prediction is your goal. See for example this very nice discussion.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Cristoph, you might want to be a bit more precise regarding your second paragraph --- you can find some counterexamples for this definition of bias here: stats.stackexchange.com/questions/59369/confounder-definition/… $\endgroup$
    – Carlos Cinelli
    Nov 9, 2017 at 17:54
  • 4
    $\begingroup$ Your definition does not discredit mediating variables or colliders for adjustment; adjusting for these variables cause biases of collider bias or effect attenuation. It also operates under a closed world example, in this you must assume you have measured all possible confounding variables, an assumption which is rarely met, nor discussed. A full definition of confounding bias is rather difficult. $\endgroup$
    – AdamO
    Nov 9, 2017 at 18:02
  • 2
    $\begingroup$ Further, In inferential statistics, it is important to label "motivation" as a confounding variable per the earlier discussions. Also, your comments only apply to GLMs with linear or log links. $\endgroup$
    – AdamO
    Nov 15, 2017 at 20:30
11
$\begingroup$

the solution for OVB is to include all those predictors that control the effect of confounding covariates not all predictors for dependent variable Y.

Yes, this is correct if you are more precise about it. For identification purposes, you should include the variables that control the effect of confounding and avoid those that open confounding paths or mediate the effect you are trying to measure (if you are interested in the total effect) --- that is, you should include those variables that satisfy the backdoor criterion. You should not indiscriminately include all predictors of $Y$, if by predictor you mean anything that "predicts" $Y$ --- this could bias your estimate. You can find a gentle, example-based, introduction to the topic in this Crash Course in Good and Bad Controls.

In this same vein, it's worth noticing that Christoph's answer is not strictly correct:

an omitted variable causes bias if it is both (a) related to the outcome Y and (b) correlated with the predictor X whose effect on Y you are primarily interested in

This is not true. Correlational criteria is not necessary nor sufficient to define what a confounder is. This is a common misconception on the definition of confounders, illustrated in this other answer.

Of course, which variables to include that guarantees identification addresses only the matter of getting consistent estimates of the causal quantity of interest. You have many other problems to address, such as the efficiency of your estimate (so you might choose/avoid variables that reduce/increase variance), biases due to misspecification of the functional form etc.

Improve this answer
$\endgroup$
7
  • 4
    $\begingroup$ (+1) can you provide a lay-explanation of what the backdoor criterion is? $\endgroup$
    – AdamO
    Nov 9, 2017 at 18:07
  • 3
    $\begingroup$ @AdamO basically a set of variables that blocks the effect of common causes, that do not open new confounding paths (such as colliders, but you in some cases as in fig 3.4 of the pdf you might need to control for colliders, so you need to further block the opened path) and that do not include variables that mediate the effect you are trying to measure (if you are interested in the total effect). $\endgroup$
    – Carlos Cinelli
    Nov 9, 2017 at 18:09
  • 2
    $\begingroup$ Indeed, the sake of the request was to increase the accessibility of your (very good) answer. $\endgroup$
    – AdamO
    Nov 9, 2017 at 18:31
  • $\begingroup$ You precised that backdoor criterion deal with “total effect”. Now, in linear SCM, is correct to say that if our SCM is fully specified by only one structural equation any direct effect coincide with total? Then, no indirect effect exist? And any single structural parameter have total effect meaning? If the reply is too articulate I can open another question $\endgroup$
    – markowitz
    Sep 18, 2019 at 14:23
  • $\begingroup$ @markowitz it’s not correct, you need to explicitly say that all other variables do not cause each other—only then, by assumption you are saying there’s no indirect effect. $\endgroup$
    – Carlos Cinelli
    Sep 20, 2019 at 4:09
7
$\begingroup$

Theoretically, including all relevant predictors eliminates the omitted variable bias. However, it might not always be feasible to include all relevant explanatory variables in your regression (due to unawareness of relevant variables or lack of data).

Regarding the lack of knowledge about the omitted variable bias. There are a couple of good lectures out there on the OVB. Looking around, one of the most comprehensive lectures on the omitted variable bias might be this one:

https://economictheoryblog.com/2018/05/04/omitted-variable-bias

It also includes a section that discusses possible strategies against an omitted variable bias.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Including all variables can hurt the prediction accuracy of the model. It also can create a multi-collinearity problem . I am not sure what you mean when you say "it might not always be feasible to include all possible explanatory variables". $\endgroup$
    – Michael R. Chernick
    May 7, 2018 at 5:40
  • 1
    $\begingroup$ Sandro, this is false and the problem is even beyond accuracy. Including some predictors will cause permanent bias to your estimate, even with infinite samples, see stats.stackexchange.com/questions/59369/confounder-definition/…. $\endgroup$
    – Carlos Cinelli
    May 7, 2018 at 5:50
  • $\begingroup$ You are right. I meant all relevant explanatory variables. I adjusted my answer. Thanks. $\endgroup$
    – Sandro Salter
    May 7, 2018 at 6:03
  • 2
    $\begingroup$ @SandroSalter what do you mean by relevant? You need to be precise here. $\endgroup$
    – Carlos Cinelli
    May 7, 2018 at 6:08
4
$\begingroup$

Carlos' answer is good in that it addresses a major deficiency in regression modeling practice. The term OVB is very imprecise. Except under atypical mathematical structures, adjusting for other variables will change the effect estimated for a primary regressor. This alone does not mean all such variables should be included in a model.

The "backdoor criterion" specifically addresses confounding bias. An audience of experts will generally not accept/believe results from models which omit confounding variables from adjustment. This is for good reason. Omitted confounders have led to completely incorrect inference in large confirmatory studies, and further led to policies, drug indications, or media coverage which were costly and damaging. The preferred terminology here is confounding bias, rather than merely OVB. This applies to all types of models, including the most prevalent linear regression.

The second most prevalent (perhaps) model is logistic regression. There is another type of "bias" (perhaps) which arises from logistic models unrelated to confounding. You can change the primary effect by adjusting for variables which are uncorrelated with the primary regressor. This is because of the non-collapsibility of the odds ratio. This arises when the primary exposure has a heterogeneous distribution of covariates underlying the baseline risk of the outcome. The slope of the sigmoid which estimates the "averaged out" accumulation of risk per unit difference in a primary regressor is attenuated. This type of bias arises when the target of inference was individual level risk, rather than population averaged.

enter image description here

In general, the advice to modelers is to adjust for prognostic variables, or variables which, despite being unrelated to the primary regressor, are causally predictive of the outcome. Examples might be in a study on lung cancer and smoking, groups of participants by environmental ambient pollution. Assume for the moment no evidence suggests that differences in regionality satisfies backdoor criteria to confound the smoking-cancer relationship. However, the difference in risk for this environmental exposure substantially predicts risk of lung cancer. Adjusting for environmental exposure more finely stratifies these participants so that the apparent differences between smoking and non-smoking, and cancer risk are apparent.

A very nice description of the difference is found here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3147074/pdf/dyr041.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.