Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

I have two factors $a,b \in R^M$ which I wish to regress out of my variables $Y \in R^{M \times N}$. $b$ is a function of the variables $Y$ and is defined as:

$b = \frac{1}{N}Y1$

Regressing out the variables serially, the formula I have is:

$$\begin{eqnarray} Y_{\perp a} &=& Y - m(m^Tm)^{-1}m^TY \\ Y_{\perp a,b} &=& Y_{\perp a} - b_{\perp a}(b_{\perp a}^Tb_{\perp a})^{-1}b_{\perp a}^TY_{\perp a} \\ \text{is equivalent to:} \\ Y_{\perp a,b} &=& Y - [a,b]([a,b]^T[a,b])^{-1}[a,b]^TY \end{eqnarray}$$

Is there a formal way to prove that regressing out $[a,b]$ simultaneously to get $Y_{\perp a,b}$ is equivalent to regressing out $a$, then calculating $b_{\perp a}$ on the residual, $Y_{\perp a}$ and regressing that out to get the same $Y_{\perp a,b}$ .

I've tested this empirically in matlab and this appears to be true, but I can't come up with a formal proof.

share|improve this question
1  
Your notation is quite strange. It seems that you have dependent variable $Y$ and two independent variables $a$ and $b$. You then do the following: 1. Regress $a$ on $Y$ and get the residual $e_1$. 2. Regress $b$ on $a$ and get the residual $e_b$. 3. Regress $e_b$ on $e_1$. You want to find out how to prove that this is identical to residual when you regress $a$ and $b$ on $Y$? Am I correct? – mpiktas Sep 24 '12 at 12:50
Yes, that's correct. – Swiss Army Man Sep 24 '12 at 13:35
1  
Somebody will come up with an answer, but I suggest either writing down the equations yourself (with 2 variables it is not that hard) or looking up partitioned inverse and its implications for linear regressions. – mpiktas Sep 24 '12 at 13:49

1 Answer

up vote 1 down vote

Turns out there is actually a theorem which proves this: http://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.