Does the percentage overlap in confidence intervals matter? Basic stats question that's been bothering me. Let's say I've put out a survey question with two possible answers, and have gotten back responses that have overlapping confidence intervals (at 95%).

This means I can't yet say one answer is statistically more likely than the other. 
But how do these results differ from something like this:

In this example, there's far less overlap between the two confidence intervals. Is there some mathematical way that represents a higher likelihood of a significant result, or is this wishful thinking?
 A: A useful, but approximate, rule of thumb is that when the overlap in 95% CIs is less than half of the margin of error, the difference will be significant at $p \le 0.05$.  Margin of error is a fancy way of saying half the length of the confidence interval, or one "whisker."
When the 95% CIs are just touching or completely disjoint, the difference will be highly significant, with $p \le 0.01$.
In other words, you need a fair amount of overlap before the data cannot distinguish between the two groups.
This ROT is sufficiently accurate when both sample sizes are at least 10 and margins of error do not differ by more than a factor of two, even when the sample sizes are not equal.
A: If you have accepted the criterion that you will consider treatments different if their confidence intervals don't overlap, then that's your decision criterion... That being said, it doesn't make a lot of sense in reality to have a sharp cutoff for these decisions...  To describe the difference between treatments, you might look at some kind of effect size statistic, like the difference between means, or Cohen's d.
