# Multiple Regression Alternative

In the text on multiple regression I am reading, it gives an example of %BODYFAT as a function of HEIGHT and WAIST SIZE. It is a good example of a multiple regression because controlling for the variable of WAIST SIZE means HEIGHT has an effect on %BODYFAT.

My question is why not run a regression of %BODYFAT on WAIST SIZE/HEIGHT ratio so it becomes proportional and there is no need to control for variables.

Do simple solutions like this exist for many multiple regression analysis? If so can someone explain why multiple regression would be better than the solution I proposed above.

There is nothing stopping you from fitting the second model you described, if the theory or prior empirical research supports it. Your model covariates are limited only by your imagination and how you want to interpret the parameter estimates. However, your second model is not a "substitute" or "alternative" to multivariate modeling, as the interpretation of the parameters differ between the two. Consider the models you described above:

$PctBodyFat_i=\beta_0+\beta_1Height_i+\beta_2WaistSize_i+\epsilon_i$ (1)

and

$PctBodyFat_i=\delta_0+\delta_1WaistHeightRatio_i+\nu_i$ (2)

In Model 1, $\beta_1$ is interpreted as the expected change in $PctBodyFat$ corresponding to a one unit increase in $Height$ after controlling for $WaistSize$. Meaning, among people of the same $WaistSize$, what is the average (expected) change we see in $PctBodyFat$ when $Height$ increases by one unit. Similarly, $\beta_2$ is the expected or average change in $PctBodyFat$ corresponding to an increase in $WaistSize$, all other things equal. That is what "controlling for" means, holding all other values constant and changing only one parameter at a time. In Model 2, on the other hand, the interpretation of $\delta_1$ is, the expected change in $PctBodyFat$ corresponding to a one unit increase in waist-to-height ratio. Do you see how these two models are not equivalent?

Here's a worked example based on what you described above using Body Fat Data. I used Stata in this example. I also converted height to cm so everything would have the same unit of measurement.

. *model 1 multivariate
. reg pctbodyfat heightcm waistsize

Source |       SS       df       MS              Number of obs =     252
-------------+------------------------------           F(  2,   249) =  274.24
Model |  12090.3059     2  6045.15297           Prob > F      =  0.0000
Residual |  5488.68364   249  22.0429062           R-squared     =  0.6878
Total |  17578.9896   251   70.035815           Root MSE      =   4.695

------------------------------------------------------------------------------
pctbodyfat |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
heightcm |  -.1458789   .0319761    -4.56   0.000     -.208857   -.0829009
waistsize |   .6423569    .027589    23.28   0.000     .5880194    .6966944
_cons |  -14.31075   6.042646    -2.37   0.019    -26.21196   -2.409532
------------------------------------------------------------------------------


$Height$ and $WaistSize$ are both statistically significant at p<0.001 in this example, with a 1 cm increase in $Height$ expected to decrease $PctBodyFat$ whilst a 1 cm increase in $WaistSize$ expected to increase $PctBodyFat$.

So far so good. Now here's the output from Model 2 where a waist to height ratio was entered instead of height and weight separately:

. *model 2 ratio
. reg pctbodyfat wthratio

Source |       SS       df       MS              Number of obs =     252
-------------+------------------------------           F(  1,   250) =  214.53
Model |  8118.31801     1  8118.31801           Prob > F      =  0.0000
Residual |  9460.67157   250  37.8426863           R-squared     =  0.4618
Total |  17578.9896   251   70.035815           Root MSE      =  6.1516

------------------------------------------------------------------------------
pctbodyfat |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
wthratio |     69.829   4.767533    14.65   0.000     60.43935    79.21864
_cons |  -17.28263   2.517475    -6.87   0.000    -22.24079   -12.32447
------------------------------------------------------------------------------


Again, waist-to-height ratio is highly statistically significant in Model 2. The reason the coefficient is so huge is because waist-to-height ratio is less than 1 for most people, so a 1-unit increase in waist-to-height ratio can be expected to produce a fairly large magnitude in change for $PctBodyFat$. If you're uncomfortable with the large coefficient, you can multiply waist-to-height ratio by 100 and your model coefficient will be divided by 100 (i.e., 0.69829), but it's a matter of preference and what makes sense. What is interesting to note here is that Model 1 appears to have a better fit than Model 2 based on the adjusted R-squared values.

After fitting a regression model, we can predict outcome values for observations based on certain characteristics. Suppose we want to know what the predicted values are from each of the 2 models for people with these characteristics. $$\begin{array}{phwws} Person & Height & WaistSize & WaistHeightRatio \\ \hline A & 170 & 80 & .47058824 \\ B & 152 & 80 & .52631579 \\ \end{array}$$

From model 1, we expect the following percentage body fat for the 2 hypothetical cases above:

. margins, at(heightcm==170 waistsize==80)

Adjusted predictions                              Number of obs   =        252
Model VCE    : OLS

Expression   : Linear prediction, predict()
at           : heightcm        =         170
waistsize       =          80

------------------------------------------------------------------------------
|            Delta-method
|     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
_cons |   12.27839    .509845    24.08   0.000     11.27423    13.28254
------------------------------------------------------------------------------

. margins, at(heightcm=152 waistsize=80)

Adjusted predictions                              Number of obs   =        252
Model VCE    : OLS

Expression   : Linear prediction, predict()
at           : heightcm        =         152
waistsize       =          80

------------------------------------------------------------------------------
|            Delta-method
|     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
_cons |   14.90421   .9258657    16.10   0.000     13.08068    16.72773
------------------------------------------------------------------------------


We expect Person A to have a percent body fat of 12.3 and person B to have a percent body fat of 14.9. Now if we were to predict percent body fat for the same exact people using Model 2, we would get:

. margins, at(wthratio=(.47058824 .52631579))

Adjusted predictions                              Number of obs   =        252
Model VCE    : OLS

Expression   : Linear prediction, predict()

1._at        : wthratio        =    .4705882
2._at        : wthratio        =    .5263158

------------------------------------------------------------------------------
|            Delta-method
|     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
_at |
1  |   15.57807   .4578963    34.02   0.000     14.67625     16.4799
2  |   19.46947   .3881273    50.16   0.000     18.70506    20.23389
------------------------------------------------------------------------------


Using Model 2, both persons are expected to have higher percentage body fat than in Model 1, 15.6% and 19.5% for Person A and Person B, respectively. To summarize

$$\begin{array}{phwws} Person & Height & WaistSize & WaistHeightRatio & E(PBF|Model_1) &E(PBF|Model_2)\\ \hline A & 170 & 80 & .47058824 &12.3 &15.6 \\ B & 152 & 80 & .52631579 &14.9 &19.5 \\ \end{array}$$

Of course, as I stated above, there is no stopping you from entering your variables in the model every which way you like. It all depends on the theory, your preference, and what you want your model parameters to mean as far as your research question is concerned. If you have the relationships between the variables worked out beforehand and want to enter it that way, then great, but combining variables before entering into the model is still not an alternative to multivariate modeling.

As you say, this is a simple example. In reality, there is probably no getting around multivariate modeling. In fact, you may even lament the fact that you do not have enough variables to control for in your dataset, and you start worrying about endogeneity and other sources of bias in your model.

• Thank you for the detailed reply. Given the two examples above my intuition tells me that both should have equal predictive power on Y but they differ in values, how would the better predictive model be selected?
– Will
Commented Dec 22, 2016 at 2:31
• In a very general sense, the more covariates you throw into the model, the better the model fit would be, because you are capturing the effect of one covariate on the outcome "independent" of the other. Look for the usual model diagnostics, such as adjusted R-squared (the higher, the better, but be careful because in general, adding a covariate will increase the R-squared, hence using adjusted R-squared). Commented Dec 24, 2016 at 2:34
• As for what variables to choose, different schools of thought vary. Statisticians favour more parsimonious models, whereas econometricians tend to include all variables of theoretical importance ("theory" is very broad. Biological processes would qualify as "theory"). As your knowledge and experience in statistics increases, you will make these decisions for yourself. Commented Dec 24, 2016 at 2:36