What would it take for the omitted-variable bias from multiple omitted variables to cancel out?

Question

Let's stick to ordinary least squares linear regression for now, and assume the typical conditions for the Gauss-Markov theorem. If it is helpful to assume Gaussian errors, that's fine.

In such a setting, even if the OLS estimator is unbiased when the needed variables are included in the model, when a variable is omitted that should have been included, such an omission can bias the estimation of the coefficients of the included variables. This bias means estimates are too high or too low (on average).

If there is second omitted variable, there could be a second source of omitted-variable bias. If that bias is in the opposite direction as the bias coming from omitting the other variable, then the two biases could cancel out.

What would be the conditions for the biases arising from omitting multiple relevant variables to cancel out and give unbiased estimation of the included coefficients? If it helps, assume just one coefficient of true interest where unbiased estimation is desirable.

Answer 1

Expanding on the comment as an answer. First write the true model as

$$ Y_i = X_{1i}^T\beta_1 + X_{2i}^T\beta_2 + \epsilon_i \tag{1}$$

where $X_{1i}$ is a $K_1 \times 1$ vector of covariates (resp. for $X_{i2}$) then the reduced model is

$$ Y_i = X_{i1}^T\gamma + \nu_i $$

Note that the OLS estimator assumes (and essentially forces via orthogonal projection in the sample) that $\mathbb{E}[X_{i1}\nu_i] = 0 $ even though this is not true by our true model. Thus the OLS estimator is giving us the sample version (assuming full rank of the entire covariate vector) of

$$ \gamma = \mathbb{E}[X_{i1}X_{i1}^T]^{-1}\mathbb{E}[X_{i1}Y_i].$$

Substitute in the true model (1) for $Y_i$ above to yield that

$$ \gamma = \beta_1 + \mathbb{E}[X_{i1}X_{i1}^T]^{-1}\mathbb{E}[X_{i1}X_{i2}^T]\beta_2. $$

Thus bias is zero when $\mathbb{E}[X_{i1}X_{i2}^T]\beta_2 = \mathbf{0}_{K_1 \times 1}$, or like you say, $\beta_2$ is in the kernel of that (sort of) covariance matrix of $X_1$ and $X_2$.

Answer 2

One could assume (essentially a special case of Yashaswi's +1 answer) a structural model like $$ y=\beta_0+\beta_1x+\beta_2w_1+\beta_3w_2+u, $$ where $Cov((x,w_1,w_2)'u)=0$ and interest is on $\beta_1$ and we do not observe $w_1$ and $w_2$.

The plim (so strictly speaking this is about consistency and not so much unbiasedness, although the MC below suggests the argument may alread work quite well in samples of moderate size) of OLS of $y$ on $x$ (and a constant) only will be, as OLS is consistent for linear projection coefficients, $$ \frac{Cov(x,y)}{Var(x)}=\frac{Cov(x,\beta_0+\beta_1x+\beta_2w_1+\beta_3w_2+u)}{Var(x)}=\beta_1+\beta_2\frac{Cov(x,w_1)}{Var(x)}+\beta_3\frac{Cov(x,w_2)}{Var(x)} $$ So we would need $$ \beta_2\frac{Cov(x,w_1)}{Var(x)}=-\beta_3\frac{Cov(x,w_2)}{Var(x)} $$ for the omitted variable biases to cancel out. That can surely happen, although I struggle to further illustrate plausible scenarios under which it would do so - some combination of the covariance of the omitted variables to the included regressor and their structural coefficients offsetting each other.

Fwiw, here is a quick Monte Carlo consistent with (as $3\cdot (1/2)/1=-(-5)\cdot (3/10)/1=3/2$) the above story:

library(mvtnorm)
    
Sigma <- matrix(c(1, 0.5, 0.3, 0.5, 1, 0.25, 0.3, 0.25, 1), nrow=3, byrow=T)
beta <- c(1, 3, -5)

dgp.est <- function(n){
  X <- rmvnorm(n, mean = rep(0,3), sigma = Sigma)
  y <- X%*%beta + rnorm(n)
  coef(lm(y~X[,1]))[2]  
}

# works increasingly well for increasing sample size:
> summary(replicate(1000, dgp.est(200)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-0.2793  0.7689  0.9999  0.9950  1.2406  2.1911 

> summary(replicate(1000, dgp.est(2000)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.6422  0.9279  1.0047  1.0052  1.0817  1.4287 
> summary(replicate(1000, dgp.est(20000)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.8749  0.9751  0.9992  1.0005  1.0264  1.1314