How to prove OLS estimator still unbiased if omitted variable(s) are independent of included variables

Question

Using the usual matrix notation for Ordinary Linear Regression, suppose for $X_{n\times p}, Y_{n\times1}, B_{p\times1}$ and $e\sim MVN(0,I_{n\times n})$, we know the 'true' relationship is:

$$Y=XB+e$$

But suppose we fit a linear regression using only a subset of the true predictors, so suppose we block off $X$ and $B$ as follows:

$$X=\left[\begin{array}{cc}X_1 & X_2 \end{array} \right]$$

$$B=\left[\begin{array}{c} B_1\\ B_2\end{array}\right]$$

And we fit a regression model based on $Y\sim X_1$, estimating $B_1$.

My understanding is that $\hat{B_1}$ from this regression should be unbiased if $X_1$ is independent of $X_2$, but I don't know how to show this. Below is my work so far.

\begin{align*} \hat{B_1}&=(X_1^TX_1)^{-1}X_1^TY\\ E\left[\hat{B_1}\right]&=E\left[(X_1^TX_1)^{-1}X_1^TY\right]\\ E\left[\hat{B_1}\right]&=(X_1^TX_1)^{-1}X_1^TE\left[Y\right]\\ E\left[\hat{B_1}\right]&=(X_1^TX_1)^{-1}X_1^TXB\\ E\left[\hat{B_1}\right]&=(X_1^TX_1)^{-1}X_1^T\left[\begin{array}{cc}X_1& X_2\end{array}\right]\left[\begin{array}{c}B_1\\ B_2\end{array}\right]\\ E\left[\hat{B_1}\right]&=\left[\begin{array}{cc}(X_1^TX_1)^{-1}X_1^TX_1& (X_1^TX_1)^{-1}X_1^TX_2\end{array}\right]\left[\begin{array}{c}B_1\\ B_2\end{array}\right]\\ E\left[\hat{B_1}\right]&=\left[\begin{array}{cc}I& (X_1^TX_1)^{-1}X_1^TX_2\end{array}\right]\left[\begin{array}{c}B_1\\ B_2\end{array}\right]\\ E\left[\hat{B_1}\right]&=B_1+ (X_1^TX_1)^{-1}X_1^TX_2B_2\\ \operatorname{Bias}(\hat{B_1})&=(X_1^TX_1)^{-1}X_1^TX_2B_2 \end{align*}

I expected to be able to pick out some measure of covariance in the expression for bias, to then be able to say "aha, when there is no covariance, there is no bias!" but it doesn't seem to be so simple.

Answer 1

This is just a question of whether $X_1^\top X_2=0.$

I.e. is every column of $X_1$ orthogonal to every column of $X_2$?

In particular, if an $y$-intercept term is included in the regression involving only $X_1,$ that means one column of $X_1$ consists only of $1\text{s}.$ Let us assume that is the first column of $X_1$ (which, by the most usual convention, it would be). That would mean that if the entries in some column of $X_2$ do not add up to $0,$ then that column of of $X_2$ is not orthogonal to the first column of $X_1.$ Then the OLS estimator of the intercept term with predictors $X_1$ would be biased if the true model is $Y=XB+e.$

If all of the columns of $X_2$ are orthogonal to all of the columns of $X_1,$ then $\widehat B_1$ is unbiased.

This is similar to uncorrelatedness. Uncorrelatedness would say $\sum_{i=1}^n (x_i-\overline x)(w_i-\overline w) = 0.$ This condition is $\sum_{i=1}^n x_i w_i=0.$ But if every column of $X_2$ adds up to $0,$ so that $\overline w=0,$ then this condidion is the same as uncorrelatedness.

Thus if every column of $X_2$ is uncorrelated with every column of $X_1$ and also the sum of the entries in every column of $X_2$ is $0,$ then $\widehat B_1$ is unbiased. (This assumes the intercept term is included in $X_1.$)

Uncorrelatedness is weaker then independence. For example, if $w_1+\cdots + w_n=0,$ then $(w_1,\ldots,w_n)$ is uncorrelated with $(w_1^2,\ldots, w_n^2),$ but obviously these are not independent.