2
$\begingroup$

Basic stats question that's been bothering me. Let's say I've put out a survey question with two possible answers, and have gotten back responses that have overlapping confidence intervals (at 95%).

Two bars with overlapping confidence interval lines.

This means I can't yet say one answer is statistically more likely than the other.

But how do these results differ from something like this:

enter image description here

In this example, there's far less overlap between the two confidence intervals. Is there some mathematical way that represents a higher likelihood of a significant result, or is this wishful thinking?

$\endgroup$
  • 3
    $\begingroup$ By making mild assumptions, you actually can draw conclusions based on examining overlap or non-overlap of confidence intervals: see stats.stackexchange.com/questions/18215. You cannot, however, draw inferences about "statistical likelihood" without making very strong assumptions about prior probabilities. $\endgroup$ – whuber Aug 25 at 12:47
0
$\begingroup$

It is possible that the confidence intervals overlap, yet are significantly different. See articles here and here.

$\endgroup$
  • $\begingroup$ Your answer should explain the situation; you can use links as support but links cannot stand in as an answer. $\endgroup$ – Glen_b -Reinstate Monica Aug 26 at 2:19
0
$\begingroup$

If you have accepted the criterion that you will consider treatments different if their confidence intervals don't overlap, then that's your decision criterion... That being said, it doesn't make a lot of sense in reality to have a sharp cutoff for these decisions... To describe the difference between treatments, you might look at some kind of effect size statistic, like the difference between means, or Cohen's d.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.