# When is omitted variable bias non-zero?

I was just reading the Wikipedia page on omitted variable bias: [wiki for OVB][1], and I was confused by one of the main claims of the page, which is that the expected omitted variable bias is 0 iff the omitted variables have zero correlation with any of the regressors.

However, the equation they give for the omitted variable bias is: $$\widehat{\beta}= \beta +(X'X)^{-1}X'z\gamma + (X'X)^{-1}X'\epsilon,$$

where $$X$$ is our $$N \times p$$ matrix of p regression variables, $$z$$ is our $$N \times 1$$ omitted variable, $$\widehat{\beta}$$ are the coefficient estimates with $$z$$ omitted, $$\beta$$ are the estimates with $$z$$ included, $$\gamma$$ is the regression coefficient for $$z$$ in the full regression, and $$\epsilon$$ is the residual.

Contrary to what the article says, this equation appears to say that there will be expected omitted variable bias as long as the inner product $$X'z$$ is nonzero, NOT when they are uncorrelated. This is very different, as neither the regressors nor the omitted variable $$z$$ are assumed to be centered/zero-mean.

For example, suppose the regressors $$X$$ and omitted variable $$z$$ are Bernoulli R.V.s with $$p=.5$$. Then the term $$(X'X)^{-1}$$ is a matrix which, in expectation, has $$N$$ on the diagonals, and $$\frac{1}{4}N$$ elsewhere, while the term $$X'z$$ has expectation $$[\frac{1}{4}N,...,\frac{1}{4}N]^T$$. Clearly, their product does not have expectation 0, despite these variables all being uncorrelated.

So, is the overall takeaway of that section of the wiki article wrong? In the general case where our regressors are not centered, mean-0 variables, can't we have omitted variable bias even if the regressors are all uncorrelated?

Any help understanding this apparent inconsistency would be much appreciated.

• It is assumed that an intercept is included in $X$, in which case the two claims agree.
– Noah
Commented Jul 29 at 21:13
• Thanks! I think I understand at a high level what you're saying. I get that E[XY]=E[X]E[Y] if they are uncorrelated, but I'm still confused as to how X,Y uncorrelated -> ommitted variable bias is the 0 vector. The equation clearly states that the change in the coefficient for each regressor is equal to a term depending on $X'Y$, which is nonzero. what am I missing?
– Paul
Commented Jul 30 at 4:29

This sort of problem is where the Nike solution works: just do it

> data<-data.frame(x=rbinom(10000,1,.5),z=rbinom(10000,1,.5))
> data$y<-1+data$z+data\$x+rnorm(10000)
>
x z        y
1 0 1 2.106539
2 0 0 1.261558
3 1 0 1.956873
4 1 1 3.947031
5 1 0 3.357888
6 1 0 4.008513

so that's as required.

Now

> with(data, cor(x,z))
[1] 0.004189648

so essentially uncorrelated

> lm(y~x,data=data)

Call:
lm(formula = y ~ x, data = data)

Coefficients:
(Intercept)            x
1.490        1.033

> lm(y~x+z,data=data)

Call:
lm(formula = y ~ x + z, data = data)

Coefficients:
(Intercept)            x            z
0.9909       1.0287       0.9975

and there's essentially no omitted-variable bias

However

> with(data, crossprod(x,z))
[,1]
[1,] 2527
> with(data, crossprod(cbind(1,x),z))
[,1]
5020
x 2527

which are indeed not zero

But then

> XtX<-with(data,crossprod(cbind(1,x),cbind(1,x)))
> XtXinv<-solve(XtX)
>
> XtXinv%*%with(data, crossprod(cbind(1,x),z))
[,1]
0.499899739
x 0.004189628

so while $$X^Tz$$ is not zero, $$(X^TX)^{-1}X^Tz$$ is zero!

One way to think about this is that if the means of $$X,Y$$ and $$Z$$ were all zero then uncorrelatedness would be enough. We can arrange for these means to all be zero by recentering the variables. If your model has an intercept then recentering the variables has no effect on the coefficients of $$X$$ and $$Z$$, so there must also be no bias when the means are non-zero.

• Hello, and thank you for your clear explanation. It is interesting that the matrix inversion accomplishes this -- I forgot about the role of the constant vector in that matrix inversion -- it seems that was the missing piece that causes the normalized inner product to be 0. thank you!
– Paul
Commented Aug 6 at 14:47