# When is regressing out factors serially, equivalent to a simultaneous model?

I have two factors $a,b \in R^M$ which I wish to regress out of my variables $Y \in R^{M \times N}$. $b$ is a function of the variables $Y$ and is defined as:

$b = \frac{1}{N}Y1$

Regressing out the variables serially, the formula I have is:

$$\begin{eqnarray} Y_{\perp a} &=& Y - m(m^Tm)^{-1}m^TY \\ Y_{\perp a,b} &=& Y_{\perp a} - b_{\perp a}(b_{\perp a}^Tb_{\perp a})^{-1}b_{\perp a}^TY_{\perp a} \\ \text{is equivalent to:} \\ Y_{\perp a,b} &=& Y - [a,b]([a,b]^T[a,b])^{-1}[a,b]^TY \end{eqnarray}$$

Is there a formal way to prove that regressing out $[a,b]$ simultaneously to get $Y_{\perp a,b}$ is equivalent to regressing out $a$, then calculating $b_{\perp a}$ on the residual, $Y_{\perp a}$ and regressing that out to get the same $Y_{\perp a,b}$ .

I've tested this empirically in matlab and this appears to be true, but I can't come up with a formal proof.

-
Your notation is quite strange. It seems that you have dependent variable $Y$ and two independent variables $a$ and $b$. You then do the following: 1. Regress $a$ on $Y$ and get the residual $e_1$. 2. Regress $b$ on $a$ and get the residual $e_b$. 3. Regress $e_b$ on $e_1$. You want to find out how to prove that this is identical to residual when you regress $a$ and $b$ on $Y$? Am I correct? – mpiktas Sep 24 '12 at 12:50
Yes, that's correct. – Swiss Army Man Sep 24 '12 at 13:35
Somebody will come up with an answer, but I suggest either writing down the equations yourself (with 2 variables it is not that hard) or looking up partitioned inverse and its implications for linear regressions. – mpiktas Sep 24 '12 at 13:49