Writing up findings from a summary given for a linear regression model

Question

I am new to linear regression and came across the following paper when looking for examples to practice: https://www.maths.cam.ac.uk/sites/www.maths.cam.ac.uk/files/pre2014/postgrad/mathiii/pastpapers/2012/PaperIII_37.pdf

I feel as though I would benefit from a model write-up for part (d) of the first question. In particular, what does the difference in slopes in models 1 and 2 tell us?

To address specific points in the earlier parts of the question, I assume that the command ‘I(height-165)’ subtracts the fixed value of 165 cm from each of the sample observations for height. The purpose of this procedure is to center the data around a mean height, so that we may give a physical interpretation to the intercept (but is there another reason for this?). In the context of this question, the intercept is the weight of a student of height 165 cm.

My attempt for part (d) is as follows:

The F-test performed for each of the three models has a very small p-value, indicating that there is at least one non-zero coefficient in each model. The R-squared values indicate a weak to moderate strength in the relationship between predictor and corrector for each of the three models. The p-value for each of the individual coefficients is less than 0.05, with the exception of the interaction term in model 3 (=0.73139, where the interaction is between sex and centralised height). This suggests that this predictor can be removed from the third model as it is of no statistical significance (in terms of hypothesis tests, we fail to reject H0 in the t-test for that coefficient). Omitting this predictor gives us model 2, which seems the most preferable (higher R^2 value than the first model and lower p-value for the F-test).

What seems quite puzzling is that we have a relatively large standard error for the estimate of the 'sex' coefficient in the second model. This would suggest that the coefficient isn't well estimated by the model.

As for part (e), under the assumption that the residuals are Gaussian with mean 0, I would expect the maximum and minimum residuals to be evenly spaced from 0. In this case, the maximum is more than three standard errors from 0 in every model.

For part (e), is the above what the question is looking for and what should be suggested to the nutritionist in light of this data?

The only answer I can think of is that the assumption that the errors are Gaussian is not met and the linear model may not be appropriate.

Any answers to the above highlighted questions are very much appreciated.

Answer 1

I can attempt to answer most of your questions. I think when comparing lm1 to lm2, we see that upon adding sex (a statistically significant dummy variable) we see the coefficient (slope) of the demeaned height decrease. This leads me to believe that lm1 was overestimating the marginal effect of height on weight. That is, given lm1, the nutritionist might be led to believe that height has a larger effect on weight than it actually does.

As for the F-test, it may be more proper to say that the explanatory (independent) variables are jointly significant, as opposed to at least one being significant (even though that is the null being tested explicitly. T tests generally indicate individually significant variables whereas the F-test tests joint hypotheses. I wouldn't worry about the standard error on Sex as the variable takes either a 1 or 0.

And as for part (e), I agree with you that the Gaussian assumptions are not completely met. In particular, I think the error term is heteroskedastic. In classic linear regression models, we assume that the Gaussian assumptions are met, one of them is that the errors are homoskedastic (the residuals in the model are same across observations). Seeing that the max and min of the residuals are quite different, this implies that extent or magnitude of the residuals above the regression line is different from those below. There are statistical tests for this, but if the question is asking just based on that output, that is what I would see. Here is another post about heteroskedasticity that may be clearer.

I am unsure of how to compare to the residual standard errors, but I would suggest that there is heteroskedasticity present in the data and that the current models are insufficient.