Proof that estimator in overfitted model is still unbiased Assume that the true population model is given by $y=x'\beta+\epsilon$, where $x$ and $\beta$ are k-dimensional vectors, and suppose that when performing a linear regression, we accidentally overfit the model as: $$y=X\beta+Z\gamma+\epsilon,$$
where $y$ and $\epsilon
$ are now n-dimensional vectors, (i.e. sample size is $n$), $X$ and $Z$ are regressor matrices where $Z $ is redundant, and $\beta$ and $\gamma$ are conformable parameter vectors.
I am attempting to show that the overfitted estimator for $\beta$, called $b_{ofit}$, is still unbiased for $\beta$, but I don't know how to interpret the value for $y$. I see two possibilities:
The first:
$$E[b_{ofit}]=E[(X'X)^{-1}X'y]=E[(X'X)^{-1}X'(X\beta+Z\gamma+\epsilon)]  $$
The second:
$$E[b_{ofit}]=E[(X'X)^{-1}X'y]=E[(X'X)^{-1}X'(X\beta+\epsilon)]  $$
I interpret the first as what we imagine to be the expected value of $b_{ofit}$, given our overfitted model, but the second is the true expected value of $b_{ofit}$.
Can someone explain what's happening here? Should I assume that the two are equal, i.e. that $\gamma$ has zero expectation?
 A: The two expressions you write both follow the wrong logic for the problem you describe. The first tells us what happens if we estimate $y_i=X_i \beta +\epsilon_i$ when the true model is $y_i=X_i\beta + Z_i \gamma + \epsilon_i$, as you replaced $y$ with the latter, but your regressor matrix is still just $X$.  The second approach is estimating the true model correctly. 
Correct approach: 
Notice that our regressor matrix is $[X Z],$ while the true value of $y$ is given by $y_i=X_i \beta +\epsilon_i$. Therefore:
$$\left[ \begin{array}{cc} \hat{\beta}_{overfit}\\  \hat{\gamma} \end{array} \right]=\left[ \begin{array}{cc} X'X& X'Z \\ Z'X & Z'Z \end{array} \right] ^{-1} \left[ \begin{array}{cc} X'\\  Z'\end{array} \right](X\beta+\epsilon) = \left[ \begin{array}{cc} X'X& X'Z \\ Z'X & Z'Z \end{array} \right] ^{-1} \left[ \begin{array}{cc} X'(X\beta+\epsilon)\\  Z'(X\beta+\epsilon) \end{array} \right].$$
If we write 
$$\left[ \begin{array}{cc} X'X& X'Z \\ Z'X & Z'Z \end{array} \right] ^{-1}=\left[ \begin{array}{cc} A & B \\ C & D \end{array} \right],$$
it follows
$$\hat{\beta}_{overfit}=AX'(X\beta+\epsilon)+BZ'(X\beta+\epsilon).$$
Taking expectations, and using the fact that $E(\epsilon|X,Z)=0$, we get 
$$E(\hat{\beta}_{overfit})=AX'X\beta+BZ'X\beta.$$
Using the rules of block matrix inversion, we get
$$A=(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}$$
$$B=-(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}X'Z(Z'Z)^{-1}Z'.$$
It follows
$$E(\hat{\beta}_{overfit})=(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}X'X\beta  -(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}X'Z(Z'Z)^{-1}Z'X\beta.$$
Combining terms, we get
$$E(\hat{\beta}_{overfit})=(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}(X'X-X'Z(Z'Z)^{-1}Z'X)\beta.$$
Using the fact that $(X'X-X'Z(Z'Z)^{-1}Z'X)^{-1}(X'X-X'Z(Z'Z)^{-1}Z'X)$ is simply an identity matrix, we get
$$E(\hat{\beta}_{overfit})=\beta.$$
