Interpretation of results from linear model predictions

Question

I couldn't see a question addressing this. I think I'm just off the mark somewhere with my thinking. Here is a simple model:

mod <- lm(hp ~ factor(cyl), data=mtcars)
summary(mod)

             Estimate Std. Error t value Pr(>|t|)    
(Intercept)     82.64      11.43   7.228 5.86e-08 ***
factor(cyl)6    39.65      18.33   2.163   0.0389 *  
factor(cyl)8   126.58      15.28   8.285 3.92e-09 ***

This tells me the hp is significantly higher in the 6 cylinders than the baseline (4 cylinders).

Now when I run the following, I'm confused by the results.

predict(mod, data.frame(cyl=as.factor(c(4,6,8))), interval="c")

        fit       lwr      upr
1  82.63636  59.25361 106.0191
2 122.28571  92.97388 151.5975
3 209.21429 188.48769 229.9409

The fit for the first is the same as the intercept, makes sense. But the upr for the first is higher than the lwr for the second, as in, the intervals overlap. When I see confidence intervals that overlap I think, 'not significantly different'.

I don't pretend to know EVERYTHING about the ins and outs of modeling, but I thought I had a pretty solid understanding of the basics, yet I'm clearly missing something as to why these intervals overlap...

Edit: Re the confusion around the terminology of 'predicting', I merely use the word predict as that is (explicit) in the function. Am aware I am using it in the 'expansion for confidence of estimation' framework.

Note, an idea of the data:

> aggregate(hp ~ cyl, data=mtcars, mean)
  cyl        hp
1   4  82.63636
2   6 122.28571
3   8 209.21429
> aggregate(hp ~ cyl, data=mtcars, sd)
  cyl       hp
1   4 20.93453
2   6 24.26049
3   8 50.97689
> aggregate(hp ~ cyl, data=mtcars, length)
  cyl hp
1   4 11
2   6  7
3   8 14

Addition:

Based on first answer and a conversation in chat. I get that the p-value 0.039 is for whether the coefficient for 6cyl is different to 0. That would say, yes, the hp is 6cyl cars is different to that of 4cyl cars. Correct interpretation right? And then you say, "okay, so what is your estimate of the hp of a 4cyl car and the hp of a 6cyl car, please include your confidence", and then you get the second set of results, 60-106 for 4cyl and 92-151 for 6cyl. But they overlap, so how can you say they're different?

So on the one hand we say the estimate of the coefficient for 6cyl cars is sufficiently far away from that of 4 cyl cars to call them different, and on the other hand we calculate a predicted value for 4cyl and 6cyl cars and they overlap therefore are not different?

Answer 1

Your use of the confidence intervals associated with the individual parameters is incorrect.

To check if two parameters are significantly different from each other you cannot look if the confidence intervals for each parameter overlap. You have to check if the confidence interval associated with the difference between the two parameters contains $0$ or not.

More formally, you are checking if your hypothesis that $\beta_{4cl} = \beta_{6cl}$. The appropriate test statistic for the null hypothesis is: $t = \hat{\beta}_{4cl} - \hat{\beta}_{6cl}$ and thus the confidence interval is dependent on the standard error associated with this test statistic.

Answer 2

I don't really see the conflict here. Your $\beta_{6cyl}$ is tested against 0. Your prediction interval is giving you joint confidence ($\beta_0+\beta_6cyl$). So since the interval (93,151) doesn't contain 83, they're saying the same thing.

I think you're less confused than you think you are. The only issue here is that it's not about whether one CI overlaps the other, but whether the one point estimate is contained within the other's confidence interval.

Answer 3

Okay. So the penny finally dropped when talking to a colleague about it. For the 'predicted' (read expanded) confidence intervals to NOT overlap, there would essentially be 4 SE's worth of difference, since each CI is the estimate +- 2SE. And of course 4SE would be fare more significant that 0.04.

Of course this was just a simplified example of something more complex I was working on and I was looking for the best way to explain this to non statisticians, who, like I naively feel into that trap, see overlying confidence intervals and think they mustn't be different.

Further reading available: http://www.sciencedirect.com/science?_ob=MiamiImageURL&_cid=272610&_user=554529&_pii=S0741521402000307&_check=y&_origin=&_coverDate=31-Jul-2002&view=c&wchp=dGLzVBA-zSkWz&md5=f47e0bb21d5a572a7e0c3299cd0796b7/1-s2.0-S0741521402000307-main.pdf