0
$\begingroup$

I am trying to understand confidence intervals better.

If two different labs ran the same experiment, with the same number of samples and both got the same mean values but their 95% CI differed.

Lab 1 got a mean value of 1.6 & 95% CI: 1.0 to 2.2

Lab 2 got a mean value of 1.6 & 95% CI: 1.5 to 1.7

What could you actually deduce from this?

If you took both results separately, would you be any more confident that the true mean was 1.6 from the results from lab 2? I feel like I am not really understanding what a CI is and misinterpreting it.

$\endgroup$
  • 3
    $\begingroup$ @mkt I suspect you intended to write the opposite of some of the things you did. First, it is clear that the variance was much smaller in the Lab 2 sample than in the Lab 1 sample. Second, your reference to "probability density" sounds like you are taking these to be Bayesian credible intervals rather than confidence intervals, because it isn't clear to what "probability density" actually refers. As such, your comment might confuse the issue more than clarify it. $\endgroup$ – whuber Jul 17 '17 at 13:53
  • 2
    $\begingroup$ This is a problematic question because although you can deduce something about the mean by combining these CIs, something else unrelated to CIs is going on. Because one interval is so much narrower than the other, you have produced significant evidence that the variances observed by the labs differ. That would make one hesitate to interpret the CIs (or to combine them) until the reason for this difference in variances is better understood. $\endgroup$ – whuber Jul 17 '17 at 14:11
  • $\begingroup$ As whuber said, in your case it means that there is a significant difference in variance between the samples. A rule of the thumb is that, when the C.I. of the samples do not overlap, then there is a significant difference between the sample statistics (not your case). Another one is that, when a C.I. contains 0 in its range (i.e. one of the intervals is negative), then you simply don't have good enough data to estimate an interval at that confidence level. $\endgroup$ – Digio Jul 17 '17 at 14:27
  • 1
    $\begingroup$ @whuber Fair points, that was sloppy writing. I am deleting my previous comment. $\endgroup$ – mkt Jul 17 '17 at 14:48
  • $\begingroup$ @Digio Please be careful, lest you confuse readers even more. Although it's correct that non-overlap of CIs often implies significance, the inverse is not true. See stats.stackexchange.com/questions/18215. Moreover, a CI has nothing whatsoever to do with containing zeros or not: you must be thinking of using a CI in a particular kind of null hypothesis test, one that is not germane to this discussion. Your characterization of a CI is disputed by almost everyone; in particular, (1) is misleading and (2) is strikingly false. $\endgroup$ – whuber Jul 17 '17 at 17:35
1
$\begingroup$

Confidence intervals are tricky things, but your intuition is right. The fact that lab 2 gets a smaller interval means that they are more confident that the true mean is close to 1.6. The tricky part is what confidence actually means in this case.

What it does NOT mean is that there is a 95% probability that the true mean falls within this interval. What it does mean is that 95% of the intervals constructed in this way will contain the true mean. Whether or not your interval is one of those 95% or one of the unlucky few 5% one cannot say.

If a confidence interval gets smaller it means that it is "harder" to hit the true mean with 95% of these intervals, therefore if you still hit the mean in 95% of the cases with a smaller interval you have a more precise measurement. The usual way to achieve this is by increasing your sample size.

I personally find this video from Dr. Nic explains it rather well.

$\endgroup$
  • $\begingroup$ +1. Though I would say that both interpretations are generally acceptable depending on the branch of statistics. $\endgroup$ – Digio Jul 17 '17 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.