How is the omitted variable bias formula derived?

I see it is often quoted that the omitted variable bias formula is

$$\text{Bias}\left(\widehat{\beta_1}\right) = \beta_2 \cdot \text{Corr}\left(X_2,X_1\right)$$

where $$\widehat{\beta_1}$$ is the estimated coefficient in the biased model, $$\beta_2$$ is the true coefficient of the omitted variable $$X_2$$ in the full model.

I am wondering how this is derived generally. Thanks.

• A little units analysis suggests this formula is not correct, but it will be if we assume these are standardized coefficients. After all, the units of measurement of $\hat\beta_1$ and $\beta_2$ usually differ (the first is in units of the response per unit of $X_1$ while the second is units of response per unit of $X_2$) but a correlation is unitless. Thus, your equation can possibly hold only in circumstances where $X_1$ and $X_2$ have the same units, which would not be a general formula.
– whuber
Apr 10, 2023 at 19:17
• @whuber Thank you, that makes sense. Apr 10, 2023 at 19:24

Here's an example of how one analyzes such a situation. Suppose that the model

$$E[Y\mid X_1,X_2] = \beta_1 X_1 + \beta_2 X_2$$

holds where $$X_1,$$ $$X_2,$$ and $$Y$$ are random $$n$$-vectors. If you omit the second variable and use the wrong model (omitting the $$X_2$$ variable)

$$E[Y\mid X] = \gamma_1 X_1,$$

we may ask what error you might expect in using an estimate of $$\gamma_1$$ to estimate $$\beta_1.$$ When you use ordinary least squares regression to estimate $$\gamma_1,$$ the formula is

$$\hat\gamma_1 = \frac{Y\cdot X_1}{X_1\cdot X_1}.$$

Using the correct model formulation you may compute

$$E[\hat\gamma_1\mid X_1, X_2] = E\left[\frac{Y\cdot X_1}{X_1\cdot X_1}\mid X_1,X_2\right] = E\left[\frac{(\beta_1 X_1 + \beta_2 X_2)\cdot X_1}{X_1\cdot X_1}\mid X_1,X_2\right].$$

Basic properties of expectation (linearity) and conditional expectation (taking out what is known) allow you to simplify the right hand side to

$$E[\hat\gamma_1\mid X_1, X_2] = \beta_1 + \beta_2 \frac{X_2\cdot X_1}{X_1\cdot X_1}.$$

By definition, the (conditional) bias in an estimate is the difference between its expectation and estimand,

$$\text{bias} = E[\hat\gamma_1\mid X_1,X_2] - \beta_1 = \beta_2 \frac{X_2\cdot X_1}{X_1\cdot X_1}.$$

That is a general formula, applicable for fixed $$X_i$$ or for random $$X_i$$ where $$X_1\cdot X_1$$ is almost surely nonzero. Already the result is helpful, because it implies that when $$X_1$$ is orthogonal to $$X_2$$ (which is just a way of saying the numerator is zero), the bias is zero; and otherwise it shows that the bias is nonzero and it gives you information about its sign and magnitude.

If, additionally, you arrange for the $$X_i$$ to be standardized (which means their components sum to zero and the squares of their components sum to unity), the fraction on the right could be called the "correlation" of $$X_1$$ and $$X_2,$$ understanding this term to be a shorthand for

$$\operatorname{correlation}(X_1,X_2) = \frac{\sum_{i=1}^n X_{1i}X_{2i}}{\sum_{i=1}^n X_{1i}X_{1i}} = \frac{\sum_{i=1}^n X_{1i}X_{2i}}{1} = \sum_{i=1}^n X_{1i}X_{2i},$$

which is the Pearson correlation of standardized vectors. But please note that this is not the correlation in the sense that $$X_1$$ and $$X_2$$ might be random vectors: the bias was computed conditionally and still depends on $$X_1$$ and $$X_2.$$ If, for instance, each $$X_i$$ were an iid sequence of random values drawn from a bivariate distribution, that underlying distribution can have a correlation but it's unlikely to equal the value computed in the bias formula.

• Thank you so very much for this. Apr 12, 2023 at 7:01