Finding Bias$(s^2)$ in incorrect linear model

Question

I am unable to find the bias of the sample variance estimator in this problem. Unfortunately I keep coming up with

Question: Suppose that the true linear model is

$y = X_1\beta_1 + X_2\beta_2 + e$,

where $e \sim MVN(0,\sigma^2 I)$, $X_1: N \times p_1, \beta_1: p_1 \times 1, X_2: N \times p_2, \beta_2: p_2 \times 1$. But we wrongly fit the model $y = X_1\beta_1+e$ and calculate the estimators of $\beta_1$ and $\sigma^2$ as:

$\hat{\beta}_1 = (X_1'X_1)^{-1}X_1'y$,

$s^2 = \large \frac{y'[I-X_1(X_1'X_1)^{-1}X_1']y}{N-p_1}$.

(a) Show that, in general, $\hat{\beta}_1$ is a biased estimator with

Bias$(\hat{\beta}_1) = (X_1'X_1)^{-1}X_1'X_2\hat{\beta}_2$

Under what condition on $X_1$ and $X_2$ is $\hat{\beta}_1$ unbiased?

(b) Show that, in general, $s^2$ is a biased estimator with

Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$

Is the bias positive or negative? Under what condition on $\beta_2$ is $s^2$ unbiased?

Attempt:

For part (a):

Bias$(\hat{\beta}_1) = E((X_1'X_1)^{-1}X_1'y - \beta_1)$

$\hspace{15mm}= (X_1'X_1)^{-1}X_1'E(y) - \beta_1$

$\hspace{15mm}= (X_1'X_1)^{-1}X_1'(X_1\beta_1 + X_2\beta_2) - \beta_1$

$\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_1\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2) - \beta_1$

$\hspace{15mm}= I\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$

$\hspace{15mm}= \beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$

$\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_2\beta_2$

If $X_1$ and $X_2$ are orthogonal, then $\hat{\beta}_1$ is unbiased.

For part (b), I am unable to figure out how to derive:

Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$

I can get

$\frac{1}{N-p_1}\beta_1'X_1'+\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_1\beta_1+X_2\beta_2$

after taking the expectation of y on both sides.

Any help would be much appreciated.

Answer 1

Matrix notation. The sum of squared residuals is

$$\hat e' \hat e = (y-X_1\hat \beta)'(y-X_1\hat \beta) = y'M_1y$$

where $M_1 = I-X_1(X_1'X_1)^{-1}X_1'$, symmetric and idempotent.

Expand $y$ using the true model: $$\hat e' \hat e = (X_1\beta_1 + X_2\beta_2 + e)'M_1(X_1\beta_1 + X_2\beta_2 + e)$$

Now, it is easily verified that $X_1'M_1 = M_1X_1=0$. So we are left with

$$\hat e' \hat e = \beta_2'X_2'M_1X_2\beta_2 +\beta_2'X_2'M_1e+e'M_1X_2\beta_2 +e'M_1e $$

Taking the conditional expected value while using the mean-independence of the error term w.r.t. the regressors, $E(e\mid X_1, X_2) = 0$, we have

$$E(\hat e' \hat e \mid X_1, X_2) = \beta_2'X_2'M_1X_2\beta_2 + E(e'M_1e \mid X_1, X_2)$$

$$\Rightarrow E(\hat e' \hat e \mid X_1, X_2) = \beta_2'X_2'M_1X_2\beta_2 +\sigma^2\cdot \text{trace}(M_1)$$

For the fact that $E(e'M_1e \mid X_1, X_2) = \sigma^2\cdot \text{trace}(M_1)$ consult e.g Hayashi (2000) ch. 1 p. 31, chapter 1 is legally downloadable.

But also (same reference), $\text{trace}(M_1) = N-p_1$ so

$$\Rightarrow E(\hat e' \hat e \mid X_1, X_2) = \beta_2'X_2'M_1X_2\beta_2 +\sigma^2\cdot (N-p_1)$$

The bias of $s^2$ then follows immediately.