# Does the percentage overlap in confidence intervals matter?

Basic stats question that's been bothering me. Let's say I've put out a survey question with two possible answers, and have gotten back responses that have overlapping confidence intervals (at 95%).

This means I can't yet say one answer is statistically more likely than the other.

But how do these results differ from something like this:

In this example, there's far less overlap between the two confidence intervals. Is there some mathematical way that represents a higher likelihood of a significant result, or is this wishful thinking?

• It is possible that the confidence intervals overlap, yet are significantly different. See articles here and here. Aug 25, 2019 at 11:57
• By making mild assumptions, you actually can draw conclusions based on examining overlap or non-overlap of confidence intervals: see stats.stackexchange.com/questions/18215. You cannot, however, draw inferences about "statistical likelihood" without making very strong assumptions about prior probabilities.
– whuber
Aug 25, 2019 at 12:47
• I think there's confusion over estimation versus testing here. Also, the participants are asked to choose between 2 options? If so, there is a constraint a + b = 1, which makes the plots quite misleading in my opinion Dec 16, 2020 at 9:25

A useful, but approximate, rule of thumb is that when the overlap in 95% CIs is less than half of the margin of error, the difference will be significant at $$p \le 0.05$$. Margin of error is a fancy way of saying half the length of the confidence interval, or one "whisker."
When the 95% CIs are just touching or completely disjoint, the difference will be highly significant, with $$p \le 0.01$$.