# What does plotting residuals from one regression against the residuals from another regression give us?

I am working with the dataset of some heights and weights at different ages. My professor wants me to plot the residuals from regression of soma.WT9 against the residuals from regression of HT9.WT9 (don't mind the notation it's just two columns where soma is being regressed on WT9 and HT9 being regressed on WT9).

What is the purpose of this plot.

-
You are welcome to ask questions like these, but adding details about what are the variables (not only variable names and tittle details) help. It will help others with similar questions in the future. –  suncoolsu Feb 17 '11 at 7:01

This sounds like what I call an "added variable" plot. The idea behind these is to provide a visual way of whether adding a variable to a model (ht9 in your case) is likely to add anything to the model (soma on wt9 in your case).

It was explained to me like this. When you fit a linear regression, the order of the variables matters. It's kind of like imagining the variance in the soma variable as an "island". The first variables "claims" a portion of the variance on the island, and the second variable "claims" what it can from what is left over.

So basically this plot will show you if "what is left to explain" in soma's variation (residuals from soma.wt9) can be explained by "the capacity of ht9 to explain anything over and above wt9" (residuals from ht9.wt9).

You can also show mathematically what is going on. Residuals from soma.wt9 are calculate as: $$e_{i}=soma-\beta_{0}-\beta_{1}wt9$$ residuals from ht9.wt9 are: $$f_{i}=ht9-\alpha_{0}-\alpha_{1}wt9$$ Regression of $e_i$ on $f_i$ through the origin (because $\overline{e}=\overline{f}=0$, so line will pass through origin) gives $$e_{i}=\delta f_{i}$$ Substituting the residual equations into this one gives: $$soma-\beta_{0}-\beta_{1}wt9=\delta (ht9-\alpha_{0}-\alpha_{1}wt9)$$ Re-arranging terms gives: $$soma=(\beta_{0}-\delta\alpha_{0})+(\beta_{1}-\delta\alpha_{1})wt9+\delta ht9$$

Hence, the estimated slope (using OLS regression) will be the same in the model with $soma = \beta_0+\beta_{wt9}wt9 + \beta_{ht9}ht9$ as in the model $resid.soma=\beta_{ht9} resid.ht9$ This also shows explicitly why having correlated regressor variables ($\alpha_{1}$ is a rescaled correlation) will make the estimated slopes change, and possibly be the "opposite sign" to what is expected.

I think this method was actually how Multiple regression was carried out before computers were able to invert large matrices. It may be quicker to invert lots of $2\times 2$ matrices than it is to invert one huge matrix.

-

Judging by the details and variable names, soma.WT9 and HT9.WT9, you are obtaining the residuals by first regressing, soma on WT9 and HT9 on WT9 (right?). If I understood you correctly, the scatter plot between soma.WT9 and HT9.WT9 will tell you -- if after removing the effects of WT9 (possibly linear effects in your case) from HT9 and soma, is there a relationship between HT9 and soma. This is beneficial in the case when WT9 explains all the source of variation in soma, then the scatter plot between soma.WT9 and HT9.WT9 will not show any particular (recognizable/standard) pattern. It may be also called partial residual plots.

-
Well, actually this is the study of “Physical Growth of California Boys and Girls from Birth to Eighteen Years” by Read D. Tuddenham and Margaret M. Snyder back from 1954. The idea was that people in this study misused statistics and hence, made false inferences about what the dataset was supposed to show. Plotting the residuals was one way of showing it. Now, let me define some of the variables: regression soma.WT9 represents the regression of so called somatotype on weight at nine years of age. and regression of HT9.WT9 represents the regression of height at 9 years of age onto weight at 9yr –  GKED Feb 17 '11 at 7:33
further, i then take the residuals from both of those regressions and plot them against each other. –  GKED Feb 17 '11 at 7:35
Unfortunately that's not available online... Could you just tell us what 'soma' is? –  onestop Feb 17 '11 at 7:35
@onestop: the data is in the R package alr3 as BGSall BGSgirls BGSboys (see rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=alr3:BGSall) and "soma" seems to be "Somatotype, a 1 to 7 scale of body type" developed by William Herbert Sheldon (described in en.wikipedia.org/wiki/Somatotype_and_constitutional_psychology) –  Henry Feb 17 '11 at 8:17