below is an exercise that is really giving me a hard time, I believe that there is a simple way around it but I can not find it:

Assume the correct regression model is Y = X$\beta$ + $\epsilon$ for E($\epsilon$) = 0 and var($\epsilon$) = $\sigma^2$I.

Assume the matrix X of dimensions n × m with m < n has full rank. Denote by $\hat\beta$ the ordinary least squares estimator of $\beta$. Assume as known that the upper left corner of the inverse of $$\begin{bmatrix}\Sigma_{11}&\Sigma_{12}\\\Sigma_{21}&\Sigma_{22} \end{bmatrix}$$

$\Sigma_{11} = \Sigma_{11}^{-1}+\Sigma_{11}^{-1}\Sigma_{12}(\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12})^{-1}\Sigma_{21}\Sigma_{11}^{-1}$

and the lower right corner is $\Sigma_{22} = (\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12})^{-1}$

a. Assume that we forget some independent variables and fit the regression model $Y = X_1\beta_{1\ast}+\epsilon_{\ast}$

where X = [X1; X2] and E($\epsilon_{\ast}$) = 0 and var($\epsilon_{\ast}$) = $\sigma^2$I.

Write $\beta$=$\begin{bmatrix}\beta_{1}\\\beta_{2}\end{bmatrix}$

Assuming the “wrong” model we estimate $\beta_{1}$ by $\hat\beta_{1\ast}$=$(X_1^TX_1)^{-1}X_1^TY$

Let $\hat\beta_1$ be the best unbiased linear estimator of $\beta_1$ in the correct model. Show that var($\hat\beta_1$) − var($\hat\beta_{1\ast}$) = $AB^{-1}A^{T}$

where A = $(X_1^TX_1)^{-1}X_1^TX_2$


Now, my first idea was to express the $\hat\beta_1$ from the partitioned model, and using Takeshi Amemiya textbook,


Where $M_2=I-X_2(X_2^TX_2)^{-1}X_2^T$

Where $M_2$ is idempotent, and $M_2X_2=0$

Therefore I finally get $\hat\beta_1$=$\beta_1$+$(X_1^TM_2X_1)^{-1}X_1^TM_2\epsilon$. The expected value of $\hat\beta_1$ is $\beta_1$ because the expected value of $\epsilon$ is 0, and the term that multiplies the $\epsilon$ is nonstochastic.

To calculate the variance of $\hat\beta_1$ I take the formula E(($\hat\beta_1$-E($\hat\beta_1$)($\hat\beta_1$-E($\hat\beta_1)^T$) which gets me to $\sigma^2(X_1^TM_2X_1)^{-1}$.

For $\hat\beta_{1\ast}$ I plug in the 'wrong' model and get $\beta_{1\ast}$+$(X_1^TX_1)^{-1}X_1^T\epsilon_{\ast}$. I get the expected value of $\hat\beta_{1\ast}$ to be $\beta_{1\ast}$ because the expected value of $epsilon_{\ast}$ is zero, so the variance I get using the same approach as above is $\sigma^2(X_1^TX_1)^{-1}$.

Now, when I subtract the variances as requested in the exercise, I don't get nearly the same thing they got in the text, I still have $\sigma^2$, and I can't get rid of the bracket because the inverse of the entire product is defined but the inverse of individual matrices might not be. Additionally, I have no idea what to do with the clue I was given about the upper left and the lower right corner of the inverse. Would there be a soul kind enough to give me a hint if I am even going in the proper direction? Thank you very very much

  • $\begingroup$ Please add the "self-study" tag to your question. $\endgroup$ – Peter Flom Feb 22 at 12:34
  • $\begingroup$ @PeterFlom Thank you for your comment, I just edited the post. $\endgroup$ – Ivana Feb 22 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.