0
$\begingroup$

I am trying to understand confidence intervals better.

If two different labs ran the same experiment, with the same number of samples and both got the same mean values but their 95% CI differed.

Lab 1 got a mean value of 1.6 & 95% CI: 1.0 to 2.2

Lab 2 got a mean value of 1.6 & 95% CI: 1.5 to 1.7

What could you actually deduce from this?

If you took both results separately, would you be any more confident that the true mean was 1.6 from the results from lab 2? I feel like I am not really understanding what a CI is and misinterpreting it.

Improve this question
$\endgroup$
12
  • 3
    $\begingroup$ @mkt I suspect you intended to write the opposite of some of the things you did. First, it is clear that the variance was much smaller in the Lab 2 sample than in the Lab 1 sample. Second, your reference to "probability density" sounds like you are taking these to be Bayesian credible intervals rather than confidence intervals, because it isn't clear to what "probability density" actually refers. As such, your comment might confuse the issue more than clarify it. $\endgroup$
    – whuber ♦
    Commented Jul 17, 2017 at 13:53
  • 2
    $\begingroup$ This is a problematic question because although you can deduce something about the mean by combining these CIs, something else unrelated to CIs is going on. Because one interval is so much narrower than the other, you have produced significant evidence that the variances observed by the labs differ. That would make one hesitate to interpret the CIs (or to combine them) until the reason for this difference in variances is better understood. $\endgroup$
    – whuber ♦
    Commented Jul 17, 2017 at 14:11
  • $\begingroup$ As whuber said, in your case it means that there is a significant difference in variance between the samples. A rule of the thumb is that, when the C.I. of the samples do not overlap, then there is a significant difference between the sample statistics (not your case). Another one is that, when a C.I. contains 0 in its range (i.e. one of the intervals is negative), then you simply don't have good enough data to estimate an interval at that confidence level. $\endgroup$
    – Digio
    Commented Jul 17, 2017 at 14:27
  • 1
    $\begingroup$ @whuber Fair points, that was sloppy writing. I am deleting my previous comment. $\endgroup$
    – mkt
    Commented Jul 17, 2017 at 14:48
  • $\begingroup$ @Digio Please be careful, lest you confuse readers even more. Although it's correct that non-overlap of CIs often implies significance, the inverse is not true. See stats.stackexchange.com/questions/18215. Moreover, a CI has nothing whatsoever to do with containing zeros or not: you must be thinking of using a CI in a particular kind of null hypothesis test, one that is not germane to this discussion. Your characterization of a CI is disputed by almost everyone; in particular, (1) is misleading and (2) is strikingly false. $\endgroup$
    – whuber ♦
    Commented Jul 17, 2017 at 17:35

1 Answer 1

Reset to default
1
$\begingroup$

Confidence intervals are tricky things, but your intuition is right. The fact that lab 2 gets a smaller interval means that they are more confident that the true mean is close to 1.6. The tricky part is what confidence actually means in this case.

What it does NOT mean is that there is a 95% probability that the true mean falls within this interval. What it does mean is that 95% of the intervals constructed in this way will contain the true mean. Whether or not your interval is one of those 95% or one of the unlucky few 5% one cannot say.

If a confidence interval gets smaller it means that it is "harder" to hit the true mean with 95% of these intervals, therefore if you still hit the mean in 95% of the cases with a smaller interval you have a more precise measurement. The usual way to achieve this is by increasing your sample size.

I personally find this video from Dr. Nic explains it rather well.

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1. Though I would say that both interpretations are generally acceptable depending on the branch of statistics. $\endgroup$
    – Digio
    Commented Jul 17, 2017 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.