# Least squres in partitioned form

Consider the following linear model: $$\label{eq:1} Y = X_1\beta_1 + X_2\beta_2 + e$$

1. Using definition of OLS estimator show the following $$\begin{bmatrix} X_1'X_1 &X_1'X_2 \\ X_2'X_1 &X_2'X_2 \end{bmatrix} \begin{bmatrix} \hat\beta_1 \\ \hat\beta_2 \end{bmatrix} = \begin{bmatrix} X_1'Y \\ X_2'Y \end{bmatrix}$$

2. Find the least squares estimate for $\beta = (\beta_1 \; \beta_2)$ subject to the constraint $\beta_2 = 0$ (using first equation). Interpret your finding. (Hint: minimize the sum of squares).

3. Deduct from the system in question 1 that $\hat\beta_1$ equals to the regression of $Y$ on $X_1$ minus a correction factor.

4. Using the correction factor, discuss two situations where $\hat\beta_1$ in the initial model is equal to the simple regression of $Y$ on $X_1$.

I was able to do the first one. I am frankly confused with the second one. Since I have a constraint $\beta_2 = 0$, then I delete it from the initial equation to obtain

$$Y = X_1\beta_1 + e$$ and show that $\hat\beta_1 = (X_1'X_1)^{-1}X_1'Y$. Is it correct?

Now the third one. Rewrite the system as \begin{aligned} X_1'X_1\hat\beta_1 + X_1'X_2\hat\beta_2 = X_1'Y \\ X_2'X_1\hat\beta_1 + X_2'X_2\hat\beta_2 = X_2'Y \end{aligned}

Express $\hat\beta_2$ from the second equation as $$\hat\beta_2 = (X_2'X_2)^{-1}X_2'Y - (X_2'X_2)^{-1}X_2'X_1\hat\beta_1$$ insert it to the first equation and after doing some algebra I get $$\hat\beta_1 = \left(X_1'X_1 - X_1'P_2X_1\right)^{-1}X_1'Y$$ where $P_2 = X_2(X_2'X_2)^{-1}X_2'$ is a projection matrix on $X_2$. However, I can't see how to show that this is equivalent to the regression of $Y$ on $X_1$ minus a correction factor. Though of course I see that this formula is similar to regressing $Y$ on $X_1$, is it possible to proceed further to come up with more "convincing" expression?

For number 4 the first possible answer is the case when $P_2X_1 = 0$, i.e. $X_2$ is orthogonal to $X_1$. What is the second one? The case when $\beta_2 = 0$?

• Can you please provide a better title for your question? The current title is not very informative. – Xu Wang Nov 14 '16 at 17:55
• @Xu wang, done . – tosik Nov 14 '16 at 18:14