control "for post-treatment" variables vs omitted variable bias

Question

in chapter 9 of gelman's data analysis using regression and multilevel/hierarchical models, page 170 presents a simple example on the bias of an omitted variable $x$ from a regression of an outcome $y$ on treatment $T$. It uses three regressions:

correct specification including $x$: $$y_i = \beta_0 + \beta_1 T_i + \beta_2 x_i + \epsilon_i$$

omitting the confounding variable:

$$y_i = \beta_0^* + \beta_1^* T_i + \epsilon_i$$

and regressing the confounding variable on the treatment:

$$x_i = \gamma_0 + \gamma_1 T_i + \nu_i$$

then it substitutes this last expression into the first regression to arrive at

$$y_i = (\beta_0 + \beta_2 \gamma_0) + (\beta_1 + \beta_2 \gamma_1) T_i + (\epsilon_i + \beta_2 \nu_i)$$

and equates $\beta_1^*$ to $\beta_1 + \beta_2^* \gamma_1$ to show that when there is association between the omitted variable and the treatment or between the omitted variable and the outcome then neither $\gamma_1$ or $\beta_2$ are, correspondingly, zero, so there is bias in the estimate of the effect of $T$.

Later, on page 188 the author advises against "controlling for variables measured after treatment / mediators / intermediate outcomes" by using an example of a home visits treatment $T$, outcome $y$ corresponding to a cognitive score on children, and parenting quality $z$.

The author argues that including $z$ in the regression of $y$ on $T$ is problematic because there is an effect of $T$ on $z$. So the coefficient of $T$ in such a regression corresponds to a comparison of units which are identical in $x$ and $z$ but differ in $T$ - but because $T$ has an effect on $z$ these units automatically have different $z$ characteristics (if $T$ is positive, then the unit that did not receive the treatment has lower underlying $z$ "parental skill" than the other in order for the two observations to have the same observed value of $z$). While this makes sense, I am having trouble reconciling this with the earlier example of omitted variable bias. Can't the same be said in the earlier example when $x$ is not omitted? i.e. including it when $T$ has an effect on it implies comparing units with different $T$ values, fixed $x$, but thus different underlying "$x$" characteristics? How are the two cases "different"?

Answer 1

The term 'bias' in 'omitted variable bias' is a bit unhelpful.

In the models $$E[Y|X]=\alpha_0+\alpha_X X$$ and $$E[Y|X,Z]=\beta_0+\beta_X X+\beta_Z Z$$ you will not in general have $\alpha_X=\beta_X$. That's a completely uncontroversial statement of linear algebra.

The difference implies that $\hat \alpha_X$ is a biased estimator of $\beta_X$, and that's where the term "omitted variable bias" came from historically. However, it's equally true that $\hat\beta_X$ is a biased estimator of $\alpha_X$, so you could also call it "included variable bias".

In old-fashioned statistics textbooks it's unfortunately common to think of model selection as about finding "the true model", and that "the true model" is defined as the largest model where all the coefficients are not zero. If you took this point of view, you would regard the larger model as better and the parameter $\beta_X$ as a better parameter and so you would talk about omitted variable bias rather than included variable bias. We don't do this nowadays because we understand that model selection isn't about a 'true model' in this sense.

In particular, if you want to interpret the coefficient of $X$ as the effect of $X$ on $Y$, you must include some $Z$s (confounders), you must exclude some other $Z$s (colliders, mediators, post-treatment variables), and it's up to you what to do with others (eg precision variables).

So: if $Z$ is a confounder for the effect of $X$ on $Y$, you want to include $Z$ in the model and estimate $\beta_X$. If it's a collider or a mediator or other post-treatment variable you want to exclude $Z$ and estimate $\alpha_X$. And the modern way to think about this is that $\alpha_X$ and $\beta_X$ are just different parameters, so that it is a bit pointless to ask whether $\hat\alpha_X$ is a biased estimator of $\beta_X$ or $\hat\beta_X$ is a biased estimator of $\alpha_X$.

Answer 2

In many of the causality texts, we classify a variable's relation to the exposure or outcome of interest according to their causal relationship as described in a directed acyclic graph (DAG). In this framework, these variables are either:

• Confounders which are causally predictive of exposure and outcome for which omitting the variable leads to biased estimates of effect.
• Precision variables in collapsible models which are unrelated to exposure but are predictive of the outcome so that adjustment for these factors reduce the residual error and improves inference and precision
• Prognostic variables in non-collapsible models which are like precision variables except that adjustment also changes the effect measure
• Mediator variables which are caused by the exposure and which shouldn't be adjusted unless the effect of the predictor be attenuated and the direct effect rather than the total effect is reported
• Colliders which are caused by the outcome and generally should not be adjusted

This framework holds in longitudinal and cross-sectional analyses. "Omitted variable bias" can only be understood in this framework, because many intrepid analysts accidentally advocate adjusting for mediators "because they're important" - meaning they predict the outcome. However, according to the theory above, the opposite is true - leave mediators out.

In a longitudinal data analysis, the lagged exposure or the lagged outcome tend to predict later variable values. As such, they become mediators, so adjustment will attenuate the estimated effect of exposure.

As an example, you might study the longitudinal use of statins versus the risk of developing congestive heart failure (CHF). Blood pressure is certainly a complicated variable, because high blood pressure at baseline can indicate novel use of statin, but development of CHF will lower blood pressure. If statin use post-baseline is adjusted, you'll likely find no relationship (or a reduced effect) between statin use and incidence of CHF after adjusting for blood pressure.