Bad Controls and Omitted Variables

Question

The traditional manner (in Economics at least) to explain an omitted variables bias involves the consideration of a Mincer type regression:$$w_{it}=\alpha+x_{it}'\beta+\gamma E_{i}+\alpha_{i}+\epsilon_{it}$$ where the LHS denotes wage of individual i at time $t$, $x_{it}'$ denotes a vector of controls , $E_{i}$ denotes education levels and $\alpha_{i}$ denotes individual specific heterogeneity, like ability, which is probably correlated with Education. As a result, if we do not “control” for ability in some manner, we obtain biased estimates of $\gamma$.

Now, I have come across some readings, in particular related to “bad controls.” What these readings point towards is that inclusions as controls of variables that could potentially themselves be outcome variables can lead to biases in parameters of interest.

Using such a line of reasoning, even if we did have a measure of ability, including it in a regression would point to this problem, because I can think of many reasons why education levels are a function of ability (the nobel prize winning model by Spence points towards exactly this hypothesis).

In an omitted variables scenario, we assume that a problem can exist if :

• $cov(.)$ between the included regressor(s) and the exluded one(s) is non 0

• The excluded regressor(s) are relevant.

This leads me to my question. If the omitted variable is supsected of having a non zero cov(.) with the included one, there are two possible scenarios:

1. One of the causes the other, leading to a dependence between the two

2. The two are caused by a third variable.

Case 2 seems to be fine, as long as this third variable is not important in determining $w_{it}$ . But Case 1 is definitely problematic. It seems to me that there may be a tradeoff between correcting for an omitted variables bias problem,and a bad control problem. How can one reconcile this?

Answer 1

There is no reason to wonder whether a variable is a "bad control" anymore. We have simple graphical criteria for deciding whether a variable should be included in the regression equation given your target query and your model. If, for instance, you want to estimate the average causal effect via regression adjustment, "good controls" are characterized by the backdoor criterion.

For instance, let me show an example where your case 2 would also be problematic. Consider the model below, where all disturbances $u$ are mutually independent standard gaussian random variables:

$$ z = u_1 + u_2 + u_z\\ x = u_1 + u_x\\ y = x + u_2 + u_y $$

Note that $z$ is correlated both with $x$ and $y$, and $z$ is not an "outcome" (it is a pretreatment variable). Yet, $z$ is "bad control" here, and adjusting for $z$ will bias your effect estimates. This happens because adjusting for $z$ opens a spurious colliding path $x \leftarrow u_1 \rightarrow z \leftarrow u_2 \rightarrow y$.

Here is a simple R code for you to see this in practice:

n <- 1e5
u1 <- rnorm(n)
u2 <- rnorm(n)
z <- u1 + u2  + rnorm(n)
x <- u1 + rnorm(n)
y <- x - 2*u2 + rnorm(n)
lm(y ~ x) # unbiased
#> 
#> Call:
#> lm(formula = y ~ x)
#> 
#> Coefficients:
#> (Intercept)            x  
#>   -0.002443     0.996894
lm(y ~ x + z) # biased, bad control
#> 
#> Call:
#> lm(formula = y ~ x + z)
#> 
#> Coefficients:
#> (Intercept)            x            z  
#>  -0.0009577    1.3976798   -0.8012717

Another interesting example is the following:

Again, here $z$ is a pre-treatment variable. But, if you naively "control" for $z$ this will amplify any existing bias. In this case, it turns out you can't obtain an unbiased estimate via adjustment, but you could recover the causal effect using instrumental variables.

Here is some R code for you to see this in practice:

n <- 1e5
z <- rnorm(n)
u <- rnorm(n)
x <- 2*z + u + rnorm(n)
y <- x + u + rnorm(n)
lm(y ~ x) # biased
#> 
#> Call:
#> lm(formula = y ~ x)
#> 
#> Coefficients:
#> (Intercept)            x  
#>     0.00338      1.16838
lm(y ~ x + z) # even more biased
#> 
#> Call:
#> lm(formula = y ~ x + z)
#> 
#> Coefficients:
#> (Intercept)            x            z  
#>    0.002855     1.495812    -0.985012

This discussion may also be helpful.