3
$\begingroup$

Let's stick to ordinary least squares linear regression for now, and assume the typical conditions for the Gauss-Markov theorem. If it is helpful to assume Gaussian errors, that's fine.

In such a setting, even if the OLS estimator is unbiased when the needed variables are included in the model, when a variable is omitted that should have been included, such an omission can bias the estimation of the coefficients of the included variables. This bias means estimates are too high or too low (on average).

If there is second omitted variable, there could be a second source of omitted-variable bias. If that bias is in the opposite direction as the bias coming from omitting the other variable, then the two biases could cancel out.

What would be the conditions for the biases arising from omitting multiple relevant variables to cancel out and give unbiased estimation of the included coefficients? If it helps, assume just one coefficient of true interest where unbiased estimation is desirable.

$\endgroup$
2
  • $\begingroup$ Suppose that $X_{1i}$ is a $K_1 \times 1$ vector of included variables and $X_{2i}$ is a $K_2 \times 1$ vector of omitted variables, then bias cancels out when $\mathbb{E}[X_{1i}X_{2i}^T]\beta_1 = \mathbf{0}_{K_1 \times 1}$, where $\beta_1$ is the parameter vector associated with $X_1$ in the "true" model. $\endgroup$ Feb 24, 2023 at 18:22
  • $\begingroup$ @YashaswiMohanty So when $\beta_1$ is in the kernel of the (expected value of the) matrix generated by $X_{1i}X^T_{2i}$, right? Want to post that as an answer? $\endgroup$
    – Dave
    Feb 24, 2023 at 18:26

2 Answers 2

5
$\begingroup$

Expanding on the comment as an answer. First write the true model as

$$ Y_i = X_{1i}^T\beta_1 + X_{2i}^T\beta_2 + \epsilon_i \tag{1}$$

where $X_{1i}$ is a $K_1 \times 1$ vector of covariates (resp. for $X_{i2}$) then the reduced model is

$$ Y_i = X_{i1}^T\gamma + \nu_i $$

Note that the OLS estimator assumes (and essentially forces via orthogonal projection in the sample) that $\mathbb{E}[X_{i1}\nu_i] = 0 $ even though this is not true by our true model. Thus the OLS estimator is giving us the sample version (assuming full rank of the entire covariate vector) of

$$ \gamma = \mathbb{E}[X_{i1}X_{i1}^T]^{-1}\mathbb{E}[X_{i1}Y_i].$$

Substitute in the true model (1) for $Y_i$ above to yield that

$$ \gamma = \beta_1 + \mathbb{E}[X_{i1}X_{i1}^T]^{-1}\mathbb{E}[X_{i1}X_{i2}^T]\beta_2. $$

Thus bias is zero when $\mathbb{E}[X_{i1}X_{i2}^T]\beta_2 = \mathbf{0}_{K_1 \times 1}$, or like you say, $\beta_2$ is in the kernel of that (sort of) covariance matrix of $X_1$ and $X_2$.

$\endgroup$
2
$\begingroup$

One could assume (essentially a special case of Yashaswi's +1 answer) a structural model like $$ y=\beta_0+\beta_1x+\beta_2w_1+\beta_3w_2+u, $$ where $Cov((x,w_1,w_2)'u)=0$ and interest is on $\beta_1$ and we do not observe $w_1$ and $w_2$.

The plim (so strictly speaking this is about consistency and not so much unbiasedness, although the MC below suggests the argument may alread work quite well in samples of moderate size) of OLS of $y$ on $x$ (and a constant) only will be, as OLS is consistent for linear projection coefficients, $$ \frac{Cov(x,y)}{Var(x)}=\frac{Cov(x,\beta_0+\beta_1x+\beta_2w_1+\beta_3w_2+u)}{Var(x)}=\beta_1+\beta_2\frac{Cov(x,w_1)}{Var(x)}+\beta_3\frac{Cov(x,w_2)}{Var(x)} $$ So we would need $$ \beta_2\frac{Cov(x,w_1)}{Var(x)}=-\beta_3\frac{Cov(x,w_2)}{Var(x)} $$ for the omitted variable biases to cancel out. That can surely happen, although I struggle to further illustrate plausible scenarios under which it would do so - some combination of the covariance of the omitted variables to the included regressor and their structural coefficients offsetting each other.

Fwiw, here is a quick Monte Carlo consistent with (as $3\cdot (1/2)/1=-(-5)\cdot (3/10)/1=3/2$) the above story:

library(mvtnorm)
    
Sigma <- matrix(c(1, 0.5, 0.3, 0.5, 1, 0.25, 0.3, 0.25, 1), nrow=3, byrow=T)
beta <- c(1, 3, -5)

dgp.est <- function(n){
  X <- rmvnorm(n, mean = rep(0,3), sigma = Sigma)
  y <- X%*%beta + rnorm(n)
  coef(lm(y~X[,1]))[2]  
}

# works increasingly well for increasing sample size:
> summary(replicate(1000, dgp.est(200)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-0.2793  0.7689  0.9999  0.9950  1.2406  2.1911 

> summary(replicate(1000, dgp.est(2000)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.6422  0.9279  1.0047  1.0052  1.0817  1.4287 
> summary(replicate(1000, dgp.est(20000)))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.8749  0.9751  0.9992  1.0005  1.0264  1.1314 
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.