When is omitted variable bias non-zero?

Question

I was just reading the Wikipedia page on omitted variable bias: [wiki for OVB][1], and I was confused by one of the main claims of the page, which is that the expected omitted variable bias is 0 iff the omitted variables have zero correlation with any of the regressors.

However, the equation they give for the omitted variable bias is: $$\widehat{\beta}= \beta +(X'X)^{-1}X'z\gamma + (X'X)^{-1}X'\epsilon, $$

where $X$ is our $N \times p$ matrix of p regression variables, $z$ is our $N \times 1$ omitted variable, $\widehat{\beta}$ are the coefficient estimates with $z$ omitted, $\beta$ are the estimates with $z$ included, $\gamma$ is the regression coefficient for $z$ in the full regression, and $\epsilon$ is the residual.

Contrary to what the article says, this equation appears to say that there will be expected omitted variable bias as long as the inner product $X'z$ is nonzero, NOT when they are uncorrelated. This is very different, as neither the regressors nor the omitted variable $z$ are assumed to be centered/zero-mean.

For example, suppose the regressors $X$ and omitted variable $z$ are Bernoulli R.V.s with $p=.5$. Then the term $(X'X)^{-1}$ is a matrix which, in expectation, has $N$ on the diagonals, and $\frac{1}{4}N$ elsewhere, while the term $X'z$ has expectation $[\frac{1}{4}N,...,\frac{1}{4}N]^T$. Clearly, their product does not have expectation 0, despite these variables all being uncorrelated.

So, is the overall takeaway of that section of the wiki article wrong? In the general case where our regressors are not centered, mean-0 variables, can't we have omitted variable bias even if the regressors are all uncorrelated?

Any help understanding this apparent inconsistency would be much appreciated.

-Paul [1]: https://en.wikipedia.org/wiki/Omitted-variable_bias

Answer 1

This sort of problem is where the Nike solution works: just do it

> data<-data.frame(x=rbinom(10000,1,.5),z=rbinom(10000,1,.5))
> data$y<-1+data$z+data$x+rnorm(10000)
> 
> head(data)
  x z        y
1 0 1 2.106539
2 0 0 1.261558
3 1 0 1.956873
4 1 1 3.947031
5 1 0 3.357888
6 1 0 4.008513

so that's as required.

Now

> with(data, cor(x,z))
[1] 0.004189648

so essentially uncorrelated

> lm(y~x,data=data)

Call:
lm(formula = y ~ x, data = data)

Coefficients:
(Intercept)            x  
      1.490        1.033  

> lm(y~x+z,data=data)

Call:
lm(formula = y ~ x + z, data = data)

Coefficients:
(Intercept)            x            z  
     0.9909       1.0287       0.9975  

and there's essentially no omitted-variable bias

However

> with(data, crossprod(x,z))
     [,1]
[1,] 2527
> with(data, crossprod(cbind(1,x),z))
  [,1]
  5020
x 2527

which are indeed not zero

But then

> XtX<-with(data,crossprod(cbind(1,x),cbind(1,x)))
> XtXinv<-solve(XtX)
> 
> XtXinv%*%with(data, crossprod(cbind(1,x),z))
         [,1]
  0.499899739
x 0.004189628

so while $X^Tz$ is not zero, $(X^TX)^{-1}X^Tz$ is zero!

One way to think about this is that if the means of $X,Y$ and $Z$ were all zero then uncorrelatedness would be enough. We can arrange for these means to all be zero by recentering the variables. If your model has an intercept then recentering the variables has no effect on the coefficients of $X$ and $Z$, so there must also be no bias when the means are non-zero.