5
$\begingroup$

I have a question about this table.

enter image description here

Why does the constant (intercept) change so dramatically from Model 1 to Model 2?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ No; N looks very much like the sample size; it's not the intercept at all. Evidently there were 40 measurements for brain size and whatever is the response here, perhaps IQ, but there are missing values for 2 observations on the extra predictors, so only 38 observations could be used in the fuller regression. The intercept is, as you say, here called the constant (which is a fine name in context). $\endgroup$
    – Nick Cox
    Commented Oct 11, 2020 at 11:41
  • $\begingroup$ Not the question, but the results appear to show that IQ depends on brain weight but not height, weight and gender -- and the researchers saw fit to submit with a full sample less than 40, and reviewers indulged that.. What biological or other thinking lies behind the analysis? How was the sample assembled at all. $\endgroup$
    – Nick Cox
    Commented Oct 11, 2020 at 11:48
  • 1
    $\begingroup$ Does this answer your question? Why is the intercept in multiple regression changing when including/excluding regressors? $\endgroup$
    – Sextus Empiricus
    Commented Oct 11, 2020 at 22:00
  • $\begingroup$ @NickCox Based on this test it is not so clear whether height & weight have no effect. The estimates for the variance of the marginal distribution of the coefficients are indeed large (suggesting insignificance). But since gender, height, and weight are correlated the variance of the parameters and individual t-tests does not say so much. Adding the coefficients to the model does increase the $R^2$ value so an ANOVA/F-test might show that also these parameters are significant (but indeed for a test with 38 people, with unclear selection protocol, it is not as meaningfull as it may seem to be). $\endgroup$
    – Sextus Empiricus
    Commented Oct 11, 2020 at 22:31
  • $\begingroup$ It is a bit of guesswork but an F score could be close to $$\frac{(0.27-0.13)/(3)}{0.73/(38-5)} \approx 2.1$$ This gives me a p-value (F-distribution with 3 and 33 df) of 0.12, so indeed, those extra three variables are not so significant (although the small sample size does not help to give a lot of power). $\endgroup$
    – Sextus Empiricus
    Commented Oct 11, 2020 at 22:40

2 Answers 2

Reset to default
14
$\begingroup$

The coefficients of each predictor are almost always going to change when you add more predictors. This is an example of the answer changing when you ask a different question.

Your software should let you fit a regression with no predictor at all. For example, if I try to predict people's weights with a regression with no predictors, then I will get the mean weight as a prediction. That will be shown as the intercept or constant.

If I then add height as a predictor, the intercept $b_0$ in

predicted weight $= b_0 + b_1$ height

is the prediction for a hypothetical someone with zero height. (Imagine plugging in height $= 0$; then the term with coefficient $b_1$ vanishes.) The intercept reported for $b_0$ in this case is going to be way outside the data and may even be returned as a negative number.

If I add an indicator say 1 if male and 0 if female, so that the model now is

predicted weight $= b_0 + b_1$ height $+\ b_2$ male

the intercept is now the prediction for a hypothetical someone with zero height and who is female (for whom male $= 0$). That will be different again, but not so much.

In general in

$\hat y = b_0 + b_1 x_1 + \cdots + b_J x_J = b_0 + \sum_{j=1}^J x_j$

the intercept $b_0$ is what is predicted when all the $x_j$ (so all $x_1$ to $x_J$) are zero. The intercept may be, in practice, an implausible or impossible value but that makes no difference to the principle. So, as the set of $x_j$s changes, so also will the intercept.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1. Excellent answer, Nick! $\endgroup$
    – Isabella Ghement
    Commented Oct 11, 2020 at 23:50
  • $\begingroup$ Great answer, so brilliantly sorted out for me, a novice, to comprehend :) bbbb $\endgroup$
    – V150
    Commented Oct 13, 2020 at 10:21
8
$\begingroup$

Nick Cox provided an excellent response and I wanted to add a more intuitive answer.

Model 1

Model 1 investigates the relationship between IQ and Brain size among subjects represented by the ones in the study, regardless of those subjects' Gender, Height and Weight.

In other words, if you imagine the target population of subjects from which the subjects in the study were selected, that population includes a mixture of subjects - some may be females, some may be males, some may have a height of 5 foot 9 inches, some may have a height of 5 foot 5 inches, etc., some may have a weight 160 lbs, some may have a weight of 120 lbs, etc. Model 1 takes all of these subjects and studies the relationship between their IQ and Brain size ignoring (or not accounting for) their Gender, Height and Weight. In other words, Model 1 mixes all of these subjects together and then studies the relationship of interest for the mixed subjects.

Model 2

Model 2 investigates the relationship between IQ and Brain size among subjects represented by the one in the study who have: the same Gender, the same Height and the same Weight.

For example, Model 2 investigates the relationship between IQ and Brain size among:

  • males with a height of 5 foot 9 inches and a weight of 160 pounds;
  • females with a height of 5 foot 5 inches and a weight of 120 pounds, etc.

Model 2 makes the assumption that the relationship betwewn IQ and Brain size is the same for all of these population subsets defined by combinations of values of Gender, Height and Weight supported by the ones present in your data. This relationship is called an adjusted relationship, since it is adjusted for of Gender, Height and Weight. In contrast, the relationship betwewn IQ and Brain size investigated via Model 1 is an unadjusted relationship.

Model 2 is selective about which subjects it considers - rather than mixing all subjects together, it focuses on subsets of subjects in the target population sharing the same Gender, the same Height and the same Weight.

Intercept Interpretation in Model 1

For Model 1, the (true) intercept represents the average value of IQ in the target population for those subjects for whom Brain size is equal to 0. Clearly, such subjects do not exist - if they did, they would be brainless.

Intercept Interpretation in Model 2

For Model 2, the (true) intercept represents the average value of IQ in the target population for those subjects for whom Brain size is equal to 0, Gender is equal to male, Height is equal to 0 inches and Weight is equal to 0 lbs. Again, such subjects do not exist.

Neither of the two intercepts has a realistic interpretation. If you mean-center the variable Brain size in Model 1 and the variables Brain size, Height and Weight in Model 2, you will get intercepts with a more realistic interpretation from the refitted models. Note, however, that slope coefficients in the two regression models you have here are interpretable even if the intercept has no meaningful interpretation in practice.

Intercept Interpretation in Model 1 after Mean-Centering Brain size

For the revised Model 1, the (true) intercept represents the average value of IQ in the target population for those subjects with an average Brain size.

Intercept Interpretation in Model 2 after Mean-Centering Brain size, Height and Weight

For the revised Model 2, the (true) intercept represents the average value of IQ in the target population for male subjects with an average Brain size, average Height and average Weight.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ +1. To make the point perfectly clear, I found it useful to plug typical heights and weights into Model 2 (using either gender, since its effect is so small): the result makes it obvious that the intercept can no longer be 5, because the resulting IQs would be ridiculously small. (Although we are kept in the dark here concerning the values of the "brain size" variable, it's easy to deduce a reasonable typical value from Model 1.) $\endgroup$
    – whuber ♦
    Commented Oct 11, 2020 at 21:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.