Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have two questions about confidence intervals:

Apparently a narrow confidence interval implies that there is a smaller chance of obtaining an observation within that interval, therefore, our accuracy is higher.

Also a 95% confidence interval is narrower than a 99% confidence interval which is wider.

The 99% confidence interval is more accurate than the 95%.

Can someone give a simple explanation that could help me understand this difference between accuracy and narrowness?

share|improve this question
1  
I think you mean "there is a smaller chance of obtaining an observation outside that interval". Unfortunately, a Confidence Interval may not mean what it appears to mean, due to technical, statistical issues, but in general the narrower the interval (at a given confidence level) the less uncertainty there is about the results. There are many threads on this site discussing what a Confidence Interval means (as opposed to, say, a Credible Interval). We're not even getting into Predictive Intervals... –  Wayne Sep 28 '11 at 15:21
add comment

4 Answers 4

For a given dataset, increasing the confidence level of a confidence interval will only result in larger intervals (or at least not smaller). That's not about accuracy or precision but rather about how much risk you're willing to take about missing the true value.

If you're comparing confidence intervals for the same sort of parameter from multiple data sets and one is smaller than the other, you could say that the smaller one is more precise. I prefer to talk about precision rather than accuracy in this situation (see this relevant Wikipedia article).

share|improve this answer
add comment

The 95% is not numerically attached at all to how confident you are that you've covered the true effect in your experiment. Think about the 95% not as a modifier to confidence but to interval. Perhaps "confidence interval using 95% range calculation" might be more accurate. You are allowed to say that values within the interval are likely and that you believe the true value is likely to be inside the interval and unlikely to be outside. But you really don't know how likely for your experiment.

Q1: Your first query conflates two things and misuses a term. No wonder you're confused. A narrower confidence interval is more precise but, when calculated the same way, such as the 95% method, they all have the same accuracy. They capture the true value the same proportion of the time.

Also, just because it's narrow doesn't mean you're less likely to encounter a sample that falls within that narrow confidence interval. A narrow confidence interval can be achieved one of two ways. The study could just have very low variance. The confidence interval around the boiling point of tap water at sea level is pretty small regardless of the sample size. The confidence interval around the average weight of people might be rather large because people are very variable but one can make that confidence interval smaller by just collecting more samples. In that case, as you gain more certainty about where you believe the true value is, by collecting more samples and making a narrower confidence interval, then the probability of encountering an individual in that confidence interval does go down. (OK, it goes down in any case when you increase sample size, but you may not bother collecting the big sample in the boiling water case)

The confidence interval is about trying to estimate the true mean value of the population. If you knew that spot on then you'd be even more precise (and accurate), not even have a range of estimates. But your probability of encountering a sample with that exact same value would be even lower.

Q2: A 99% confidence interval is wider than a 95%. Therefore, it's more likely that it will contain the true value. See the distinction above between precise and accurate, you're conflating the two. If I make a confidence interval narrower with lower variability and higher sample size it becomes more precise, the likely values cover a smaller range. If I increase the coverage by using a 99% calculation it becomes more accurate, the true value is more likely to be within the range.

share|improve this answer
    
The variability of the mean (=what you are trying to estimate) in the population is zero. –  Nick Sabbe Sep 29 '11 at 7:13
add comment

First of all, a CI for a given confidence percentage (e.g.95%) means, for all practical purposes (though technically it is not correct) that you are confident that the true value is in the interval.

If this is interval is "narrow" (note that this can only be regarded in a relative fashion, so, for comparison with what follows, say it is 1 unit wide), it means that there is not much room to play: whichever value you pick in that interval is going to be close to the true value (because the interval is narrow), and you are quite certain of that (95%).

Compare this to a relatively wide 95% CI (to match the example before, say it is 100 units wide): here, you are still 95% certain that the true value will be within this interval, yet that doesn't tell you very much, since there are relatively many values in the interval (about a factor 100 as opposed to 1 - and I ask, again, of purists to ignore the simplification).

Typically, you are going to need a bigger interval when you want to be 99% certain that the true value is in it, than when you only need to be 95% certain (note: this may not be true if the intervals are not nested), so indeed, the more confidence you need, the broader the interval you will need to pick.

On the other hand, you are more certain with the higher confidence interval. So, If I give you 2 intervals of the same width, and I say one is a 95% CI and the other is a 99% CI, I hope you will prefer the 99% one. In this sense, 99% CIs are more accurate: you have less doubt that you will have missed the truth.

share|improve this answer
    
thanks! so then when they say that this new research on neutrinos being faster than light has a very small confidence interval (I guess this means narrow) then that means that they are more likely to be accurate then if it was a wide confidence interval? (disregarding all other aspects) –  Dbr Sep 28 '11 at 15:02
8  
Nick, your first statement is wrong. It's not a "technical issue", it's just not correct. The confidence interval is a statement about what would happen in repeated experiments, that they would cover the true value 95% of the time. A statement about the confidence that the true value is within my given range found in my given experiment is not the same as that at all. If you removed the "that" in "that confident" and the parenthetical numerical amount then you'd be closer to the truth. You could just say that it means you believe the true value likely to fall in the interval. –  John Sep 28 '11 at 15:17
    
otherwise, the answer is pretty good... –  John Sep 28 '11 at 15:19
3  
@John: I specifically avoided saying that the interval itself is the random variable, though my sentence does not imply it not to be (admittedly, it does suggest so). I know the issues involved, but found them irrelevant for the question. I have never seen a practical situation where the difference mattered either, hence the "for all practical purposes". –  Nick Sabbe Sep 28 '11 at 15:38
3  
Haven't encountered the issue? That's like saying the p-value = the probability of the null and then saying that you've never encountered an issue with it. You won't if you stay in the right journals. It's just incorrect to say that you're 95% certain the the true value is in your current range. Treating it as some esoteric matter just means now we'll have (at least) one more person walking around saying, "I'm 95% confident the value is in this range." It would hardly change your answer to correct it. The other issues you skirt could be ignored if you changed that one statement. –  John Sep 28 '11 at 17:24
add comment

I am adding to some good answers here that I gave upvotes to. I think there is a little more that should be said to completely clear up the conclusion. I like the terms accurate and correct as Efron defines them. I gave a lengthy discussion on this very recently on a different question. Moderate whuber really liked that answer. I will not go to the same lnegth to repeat that here. However to Efron accuracy relates to the confidence level and correctness to the width or tightness of the interval. But you can't talk about tightness without considering accuracy first. Some confidence intervals are exact those are accurate because they have the actual coverage that they advertise. A 95% confidence interval can also be approximate because it uses an asymptotic distribution. Approximate intervals based on asymptotics are for a finite sample size n not going to have the advertised coverage which is the coverage you would get if the asymptotic distributionwere the exact distribution. So an approximate interval could undercover (i.e. advertise 95% when its actual coverage is only 91%) or in the rare but less serious case overcover (i.e. advertised coverage is 95% but actual in 98%). In the former case we worry about how close the actual coverage is to the advertised coverage). A measure of closeness is the order of accuracy which could be say 1/√n or 1/n. If the actual confidence level is close we call it accurate. Accuray is important with bootstrap confidence intervals which are never exact but some variants are more accurate than others.

This definition of accuracy may be different to the one the OP is referring to but it should be clear now what Efron's definition is and why it is important to be accurate. Now if you have two methods that are exact we can prefer one over the other if for any confidence level it has the smaller expected width. A confidence interval that is best in this sense (sometime called shortest) would be the one to choose. But this required exactness. If the confidence level is only approximate we could be comparing apples and oranges. One could be narrower than another only because it is less accurate and hence has a lower actual coverage than its advertised coverage.

If two confidence intervals are both very accurate or one is exact and the other very accurate comparing expected width may be okay because at least now we are looking at just two two varieties of apples.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.