What has to be true of controls (main effects and interactions) for them to be included in experimental analysis?

Consider a simple cross-sectional regression model,

$$y = \alpha + \beta \cdot T + \gamma \cdot x + \delta \cdot x \times T + \varepsilon,$$

where $T$ is a binary randomized treatment indicator, and x is a covariate (say pre-experimental outcomes, or fixed characteristics of the units). The motivation for adding $x$ is

1. to reduce variance,
2. control for any imbalance between treatment and control groups,
3. and to understand effect heterogeneity in the case of the interaction term (instead of doing subgroup analysis).

Is there a readable survey that I can refer to that explains what must be true of $x$ for my estimates of $\beta$ and $\delta$ to be valid and when this can do wrong? For example, $x$ can't be an intermediate outcome or a pre-treatment variable that is altered by expectation of being treated in the future. Variables are that directly influence the outcome $y$ but are unrelated to treatment $T$ are OK to include. Variables that influence both $T$ and $y$ need to be included.

• Just to be clear, your model is synonymous with $y = \beta_{0} + \beta_{T}T + \beta_{x}x + \beta_{Tx}Tx + \varepsilon$, yes? – Alexis Feb 24 '18 at 2:18
• Yes, that is right. – Dimitriy V. Masterov Feb 24 '18 at 2:23
• If $T$ is randomized, how can any variable but the randomization process influence it? (Also, answer edited to address causal confounding, but may reedit to add stuff about selection bias). – Alexis Feb 24 '18 at 17:09
• @Alexis Not sure if I understand your question completely. The outcome can have other causes. But that is not probably what you meant. One example is a coupon that is ineffective until a certain date. A forward-looking buyer will reduce purchases today and purchase a lot more tomorrow, which may lead to the false conclusion that inactive customers have the largest effect or even exaggerate the main treatment effect. – Dimitriy V. Masterov Feb 24 '18 at 18:05
• I think that by "Variables that influence both $T$ and $y$ need to be included." you mean "Variables that influence both the effect of $T$ on $y$ and that directly affect $y$ need to be included." Yes? Because if you instead mean "Variables that causally influence $T$," then $T$ is not randomized, because something other than the (metaphoric) coin flip of randomization is causing $T$. – Alexis Feb 24 '18 at 18:13

They have to be be predetermined -- otherwise they would constitute a "bad control." A good blog post about the topic is here, and if my memory serves there's a section on the topic in "Mostly Harmless Econometrics."

Basically if $T$ causes $x$, and $x$ causes $y$, then the coefficient on $T$ will be biased because part of its effect is being sopped up through $x$.

• Am reminded of one of my epi professors repeatedly crying out in passion in lecture "effect modifier can never be confounder!" :) – Alexis Feb 24 '18 at 2:20

I think that, in the language of formal causal reasoning, conditioning your model of $y$ on $x$ introduces bias in estimating the true causal effect of $T$ on $y$ if $x$ is a causal collider with respect to $T$ as a prior cause, and any unmeasured prior cause of both $x$ and $y$ (for any of the four combinations of the red arrows existing or not existing).

In the DAG below, $\boxed{R}$ is the randomizing process for $T$, $x$ is caused by $T$, and an unmeasured variable $U$ causes both $x$ and $y$. The red arrows indicate that these two direct causal effects of $T$ and $x$ on $y$ may, or may not exists. In any case, because $x$ is a causal "collider" in the sense described by Hernán and Robbins (and Pearl, and Greenland, and others :), conditioning your model of $y$ on $x$ will create biased estimates of the true causal effect of $T$ on $y$.

In the simplest case, where the red arrows do not exist, the true causal effect of $T$ on $y$ is zero, but conditioning on $x$ will induce an association between $T$ and $y$ (that is, you will have $\beta\ne0$), because you have "unblocked" association at the collider $x$, so association "flows" from $y \rightarrow U \rightarrow x \rightarrow T$ (see Hernán and Robbin's example and discussion on page 87).

Of course, the DAG above condenses what could be more elaborate relationships that would result in biased estimates when conditioning on $x$, because $x$ does not have to be a direct causal descendant of $T$, but may be the descended from a direct descendant of $T$ as shown here with the unmeasured variable $U_{1}$, where association "flows" from $y \rightarrow U_{2} \rightarrow x \rightarrow U_{1} \rightarrow T$:

It could also be the case that $x$ is not a causal collider, but that one of it's descendants is, as with unmeasured variable $U_{2}$—which also may or may not be a direct cause of $y$—here, where association "flows" from $y \rightarrow U_{1} \rightarrow U_{2} \rightarrow x \rightarrow T$:

See Hernán, M. A. and Robins, J. M. (2018). Causal Inference. Chapman & Hall/CRC, Boca Raton, FL. (free pre-print after the link)... particularly section 6.3 from pp73–75.

• Are any of these testable? – Dimitriy V. Masterov Feb 27 '18 at 20:34
• If $E(x|T) \ne E(x)$, I think that implies all three... I am still ruminating—well no: too frickin' busy with talks etc., but intend to get back to ruminating—on selection bias issues. But probably won't come back until Friday. Busy term. – Alexis Feb 28 '18 at 7:18