Consider two parameters with means A and B with 95% confidence intervals. If the confidence intervals of the parameters do not overlap with the mean of the other, why is it considered to be statistically significant? Please give detailed explanation with examples.

  • $\begingroup$ Your first paragraph doesn't seem to be saying the same thing as your last paragraph. Since you imply they're different ways of asking the same thing, can you make them consistent please? $\endgroup$ – Glen_b Apr 21 '16 at 9:48
  • $\begingroup$ A more detailed (but not quite so general) version of this question was asked and answered at stats.stackexchange.com/questions/18215 . It discusses how to convert the non-overlap into a confidence (which will be considerably greater than 95%). Please note that your conclusion is false in general. For it to be true, you need additional assumptions. For instance, when the parameter estimates are independent the conclusion is true. (Not all parameter estimates are independent in practice, so this concern is more than just a mathematical nicety.) $\endgroup$ – whuber Apr 21 '16 at 14:27

Two statistics are significantly different if the true difference between the two is different from 0.

$s_A$ the standard deviation of $A$ and $s_B$ the standard deviation of $B$, you can define the confidence interval for $A-B$ as

$I_{A-B} = [(A-B) - z_{0.95}\sqrt{\frac{s_A^2}{n_A} + \frac{s_B^2}{n_B}}, (A-B) + z_{0.95}\sqrt{\frac{s_A^2}{n_A} + \frac{s_B^2}{n_B}}]$

And each CI for $A$ and $B$

$I_{A} = [A - z_{0.95}\sqrt{\frac{s_A^2}{n_A}}, A + z_{0.95}\sqrt{\frac{s_A^2}{n_A}}]$

$I_{B} = [B - z_{0.95}\sqrt{\frac{s_B^2}{n_B}}, B + z_{0.95}\sqrt{\frac{s_B^2}{n_B}}]$

When the confidence interval of $A$ do not overlap with the mean $B$ then the condition is not verified and $I_{A-B}$ does not contain 0. (True mean difference can then not be 0).

As an example, $A = 10, s_A = 5, B = 15, s_B = 7, n_A = n_B = 10$

$I_A = [7.763932, 12.23607]$ $I_B = [11.8695, 18.1305]$

$I_A$ and $I_B$ do overlap but no CI overlap the means $A$ or $B$.

$I_{A-B} = [-8.847077, -1.152923]$

So the true value of $A-B$ is lesser than 0, $A$ and $B$ are different.

If $B = 13$ then

$I_B = [9.869505, 16.1305]$ ; $I_A = [7.763932, 12.23607]$

$I_A$ and $I_B$ now overlap so that $A$ is in $I_B$ but $B$ is not in $I_A$.

$I_{A-B} = [-6.847077, 0.8470768]$

So true value of $A-B$ may be 0 and thus you can not conclude that $A$ and $B$ are different.

  • $\begingroup$ This argument does not appear to be complete. What exactly do you mean by "the condition is not verified"? It is essential that you demonstrate that the difference in estimates differs from zero with at least 95% confidence, but none of these calculations appear to do so. $\endgroup$ – whuber Apr 21 '16 at 14:31
  • $\begingroup$ I thought that a CI at significance level 5% meant that if experiment was to be repeated, 95% of the time the true value for the sample would fall in the calculated interval. So calculating a CI for $A - B$ and ensuring that this CI does not contains 0 would ensure that 95% of the true difference in repeated samples would not be 0. Is there something I am understanding wrong ? $\endgroup$ – Riff Apr 21 '16 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.