Consider the following linear model: \begin{equation} \label{eq:1} Y = X_1\beta_1 + X_2\beta_2 + e \end{equation}

  1. Using definition of OLS estimator show the following \begin{equation} \begin{bmatrix} X_1'X_1 &X_1'X_2 \\ X_2'X_1 &X_2'X_2 \end{bmatrix} \begin{bmatrix} \hat\beta_1 \\ \hat\beta_2 \end{bmatrix} = \begin{bmatrix} X_1'Y \\ X_2'Y \end{bmatrix} \end{equation}

  2. Find the least squares estimate for $\beta = (\beta_1 \; \beta_2)$ subject to the constraint $\beta_2 = 0$ (using first equation). Interpret your finding. (Hint: minimize the sum of squares).

  3. Deduct from the system in question 1 that $\hat\beta_1$ equals to the regression of $Y$ on $X_1$ minus a correction factor.

  4. Using the correction factor, discuss two situations where $\hat\beta_1$ in the initial model is equal to the simple regression of $Y$ on $X_1$.

I was able to do the first one. I am frankly confused with the second one. Since I have a constraint $\beta_2 = 0$, then I delete it from the initial equation to obtain

$$Y = X_1\beta_1 + e$$ and show that $\hat\beta_1 = (X_1'X_1)^{-1}X_1'Y$. Is it correct?

Now the third one. Rewrite the system as \begin{equation} \begin{aligned} X_1'X_1\hat\beta_1 + X_1'X_2\hat\beta_2 = X_1'Y \\ X_2'X_1\hat\beta_1 + X_2'X_2\hat\beta_2 = X_2'Y \end{aligned} \end{equation}

Express $\hat\beta_2$ from the second equation as $$\hat\beta_2 = (X_2'X_2)^{-1}X_2'Y - (X_2'X_2)^{-1}X_2'X_1\hat\beta_1$$ insert it to the first equation and after doing some algebra I get $$\hat\beta_1 = \left(X_1'X_1 - X_1'P_2X_1\right)^{-1}X_1'Y$$ where $P_2 = X_2(X_2'X_2)^{-1}X_2'$ is a projection matrix on $X_2$. However, I can't see how to show that this is equivalent to the regression of $Y$ on $X_1$ minus a correction factor. Though of course I see that this formula is similar to regressing $Y$ on $X_1$, is it possible to proceed further to come up with more "convincing" expression?

For number 4 the first possible answer is the case when $P_2X_1 = 0$, i.e. $X_2$ is orthogonal to $X_1$. What is the second one? The case when $\beta_2 = 0$?

  • 1
    $\begingroup$ Can you please provide a better title for your question? The current title is not very informative. $\endgroup$ – Xu Wang Nov 14 '16 at 17:55
  • $\begingroup$ @Xu wang, done . $\endgroup$ – tosik Nov 14 '16 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.