Bodyfat, abdomen and biceps: How to interpret this data?

Question

With the help of the abdominal girth and the girth of the biceps we want to prophesy the bodyfat. To do so we collected the data of 252 men (the first 40 data are shown below). First do the linear regression of the model bodyfat/ abdomen, then the liear regression of the model bodyfat/ biceps and then the multiple regression of the model bodyfat/ (abdomen + biceps). Compare and interpret the results!

Below is a part of the data in order to give you an impression what the data look like. With R I did the two simple linear regressions and the one multiple regression and drew it (see below).

Model 1 (bodyfat/ abdomen)

For the linear model I got the two estimated coefficients $$ \hat{\theta}_1\approx-35.197,~~~\hat{\theta}_2\approx0.585. $$ With this I drew the line into the data that you see in the first picture.

Model 2 (bodyfat/ biceps)

Here, I got the two estimated coefficients $$ \hat{\theta}_1\approx-21.882,~~~\hat{\theta}_2\approx1.265. $$ Again I drew the line (s. second picture below).

Model 3 (bodyfat/ (abdomen + biceps))

Here, three coefficients had to be determined, I got $$ \hat{\theta}_1\approx -30.684,~~~\hat{\theta}_2\approx 0.645,~~~\hat{\theta}_3\approx -0.311. $$

Now the task is to compare and to interpret. But to be honest, I have some difficulties to read something in this... or to interpret something.

Maybe you can help me to compare and to interpret?

What can be seen from all this?

---

EDIT

The $R^2$-value of the first model is $R^2=0.6621$.

The $R^2$-value of the second model is $R^2=0.2431$.

The $R^2$-value of the third model is $R^2=0.6699$.

So between the abdomen and the bodyfat there is a rather good linear connection, but the linear connection between the biceps and the bodyfat seems to be rather small. Put abdomen and biceos together, there is a better linear connection which seems to be clear, because the bodyfat depends on very much factors (such as food, moving, gene, ...) and the more factors one puts into consideration, the better is the connection.

Answer 1

You seem to be on the right track regarding the R² values. One thing to consider though: R² will always increase when adding predictors to a multiple regression model; this is why we calculate adjusted R² as well. Adjusted R² may decrease when adding relatively unhelpful predictors. You can also compare the model with both predictors to the nested model with only one predictor by using an F-test, in case you're interested in doing so.

Your instructor probably also expects you to interpret the regression coefficients. Assuming $\hat\theta_1$ represents the intercept in each case, this represents the value of the dependent variable when the independent variable(s) is(/are) equal to zero. How seriously do you think you should take these values? Would you expect someone to have zero abdominal girth, let alone negative body fat?

Regardless, the intercept is useful as a starting point or adjusting value for your predictions of body fat based on abdomen and bicep girth. The slope coefficients (I'm guessing they're your $\hat\theta_{2\ \&\ 3}$) represent how much body fat changes for each unit increase of the relevant predictor. These coefficients can be positive or negative, but yours are both positive. What does this tell you about the relationships between body fat and girth of abdomen and biceps? If girth increases, how does body fat change, and by how much? You've given the answers; you just need to identify them appropriately.

For a bonus challenge, try multiplying bicep and abdomen girth and adding that product as a third predictor in multiple regression. This is an interaction term, and it might reveal some interesting info.

Answer 2

Just to add an example to underline Nick's answer. Imagine that the god of fitness, Arnold, tells us the law of fat. And it turns out to be:
$\text{Fat} = -35 + 0.6 * \text{Abdomen}$

You as a good scientist collect 252 observations, which you can simulate in R as follows:

set.seed(0) #to have the same observations
abdomen<-rnorm(n=252,mean = 85,sd=10)
fat<--35+0.6*abdomen + rnorm(252,mean=0,sd=5) 

Now your run your linear regression on it:

summary(lm(fat~abdomen))

And you get a respectable $R^2$ of .5063 and estimate the intercept at $-29.5$ and the abdomen effect at $.53$. Not perfect, but in the ballpark. And hey, they were very noisy observations!

But now your worst nemesis comes and says "biceps are also important to predict fat" and you just have to put this punk down. Well, luckily you collected also biceps information so you run one more regression:

biceps<-rnorm(n=252, mean=33,sd=4)
summary(lm(fat~abdomen+biceps))

And look at that, the $R^2$ went up a tiny little bit, it's now .5072. Is Arnold forsaking us? Now that's weird. In the code to generate fat, we didn't add biceps. What's going on? Well, as Nick already told you: the more parameters you add, the better the fit. Even though, like in this case, the new parameter was garbage.

Now, there are many ways of dealing with this. If you assume, as is correct in this case, that errors are normally distributed, you can check hypothesis of the biceps parameter to be 0. You can do cross-validation to choose between the two models. But in this case, since you posted pretty pictures, let's solve this challenge graphically.

A handy theorem is the Frisch-Waugh-Lovell theorem. It says that for multiple linear regressions, say $Y=X+Z$. Now instead of doing one regression big regression, you do two separate ones: $Y=X$ and $Z=X$, then you take the residuals from these regressions $e_1$ and $e_2$ and you regress them $e_1=e_2$. The coefficient of $e_2$ on $e_1$ is exactly the one of $Z$ on $Y$ you would have got from the original regression.

Okay, but why go such a roundabout way? Because when you plot the residuals you really plot the effect of one variable over $Y$ *after removing the effects of all the other variables$. Which is great to see the effects of each variable separately. So, if you do the FWL treatment to abdominals you get:

fatResiduals<-lm(fat~abdomen)$residuals
bicepsResiduals<-lm(biceps~abdomen)$residuals

lm(residuals~bicepsResiduals)
plot(fatResiduals~bicepsResiduals)

So you can see still a very strong linear relation of abs on fat. But if you do the same treatment to biceps:

fatResiduals2<-lm(fat~biceps)$residuals
absResiduals<-lm(abdomen~biceps)$residuals
plot(fatResiduals2~absResiduals)

It is very hard to tell if there is any relationship. The word of Arnold is true. But of course your nemesis won't believe you.

Answer 3

In the third model, you have not to sum abdomen + biceps. You have to build a multivariable model where both variables predict body fat.

To compare the models, look at the R- square values. Higher values denote better model performance. This simple comparison can be made using several fit statistics.