# What does plotting residuals from one regression against the residuals from another regression give us?

I am working with the dataset of some heights and weights at different ages. My professor wants me to plot the residuals from regression of soma.WT9 against the residuals from regression of HT9.WT9 (don't mind the notation it's just two columns where soma is being regressed on WT9 and HT9 being regressed on WT9).

What is the purpose of this plot.

-
You are welcome to ask questions like these, but adding details about what are the variables (not only variable names and tittle details) help. It will help others with similar questions in the future. –  suncoolsu Feb 17 '11 at 7:01

This sounds like what I call an "added variable" plot. The idea behind these is to provide a visual way of whether adding a variable to a model (ht9 in your case) is likely to add anything to the model (soma on wt9 in your case).

It was explained to me like this. When you fit a linear regression, the order of the variables matters. It's kind of like imagining the variance in the soma variable as an "island". The first variables "claims" a portion of the variance on the island, and the second variable "claims" what it can from what is left over.

So basically this plot will show you if "what is left to explain" in soma's variation (residuals from soma.wt9) can be explained by "the capacity of ht9 to explain anything over and above wt9" (residuals from ht9.wt9).

You can also show mathematically what is going on. Residuals from soma.wt9 are calculate as: $$e_{i}=soma-\beta_{0}-\beta_{1}wt9$$ residuals from ht9.wt9 are: $$f_{i}=ht9-\alpha_{0}-\alpha_{1}wt9$$ Regression of $e_i$ on $f_i$ through the origin (because $\overline{e}=\overline{f}=0$, so line will pass through origin) gives $$e_{i}=\delta f_{i}$$ Substituting the residual equations into this one gives: $$soma-\beta_{0}-\beta_{1}wt9=\delta (ht9-\alpha_{0}-\alpha_{1}wt9)$$ Re-arranging terms gives: $$soma=(\beta_{0}-\delta\alpha_{0})+(\beta_{1}-\delta\alpha_{1})wt9+\delta ht9$$

Hence, the estimated slope (using OLS regression) will be the same in the model with $soma = \beta_0+\beta_{wt9}wt9 + \beta_{ht9}ht9$ as in the model $resid.soma=\beta_{ht9} resid.ht9$ This also shows explicitly why having correlated regressor variables ($\alpha_{1}$ is a rescaled correlation) will make the estimated slopes change, and possibly be the "opposite sign" to what is expected.

I think this method was actually how Multiple regression was carried out before computers were able to invert large matrices. It may be quicker to invert lots of $2\times 2$ matrices than it is to invert one huge matrix.

-