1
$\begingroup$

Can it be possible that confidence intervals of two estimates intersect but the difference between estimates is statistically significant?

I can see the opposite result when CIs of two estimates do not intersect, yet the difference is not statistically significant. Consider CI(b2) includes 0 but does not include b1. As the difference usually has wider CI than b2, CI(b1 - b2) therefore includes 0.

Can anybody give an example when the difference is statistically significant indeed? Simulated calculations would be a nice illustration to the proof.

$\endgroup$
1
$\begingroup$

This was straightforward to produce an example of.

Here's a plot of two data sets with 95% CI for the population mean for each, showing substantial overlap:

Stripchart of samples, overlap in CIs is about 40% of the width of each interval

But the ordinary two sample t-test of the difference in means is significant at the 5% level (p=0.0487).

The samples:

x <- c(-0.298962792076644, 1.31710182405988, -0.0766589533299794, 
0.0337168103544878, 0.613459873853599, -0.15890274057914, -0.390659755933809, 
1.53050298362584, 0.0499823542564353, 0.466616365382981, 0.744105279446008, 
0.347365715075919, -0.390328944660933, -1.51499397031234, 0.307921692331803)

y <- c(2.24809443786109, 1.19177585960823, 0.199827407525814, 0.250899031428787, 
1.23762215395455, -0.154050289186218, 0.457502634532336, -0.783354267205565, 
1.3261971790762, 1.00561325604448, 1.40114939526361, 2.15511419519248, 
-0.211169335193589, 1.17701452262601, 0.220094909794722)

The two-sample test:

t.test(x,y,var.equal=TRUE)

        Two Sample t-test

data:  x and y
t = -2.0606, df = 28, p-value = 0.04874
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.215344491 -0.003597556
sample estimates:
mean of x mean of y 
0.1720177 0.7814887

The CI's for each sample mean (some lines omitted):

t.test(x,confint=TRUE)

95 percent confidence interval:
 -0.2392346  0.5832701
mean of x 
0.1720177 

t.test(y,confint=TRUE)

95 percent confidence interval:
 0.298467 1.264510
mean of x 
0.7814887 
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.