Are all values within a 95% confidence interval equally likely?

Question

I have found discordant information on the question: "If one constructs a 95% confidence interval (CI) of a difference in means or a difference in proportions, are all values within the CI equally likely? Or, is the point estimate the most likely, with values near the "tails" of the CI less likely than those in the middle of the CI?

For instance, if a randomized clinical trial report states that the relative risk of mortality with a particular treatment is 1.06 (95% CI 0.96 to 1.18), is the likelihood of 0.96 being the correct value the same as 1.06?

I found many references to this concept online, but the following two examples reflect the uncertainty therein:

1. Lisa Sullivan's module about Confidence Intervals states:

   The confidence intervals for the difference in means provide a range of likely values for ($μ_1-μ_2$). It is important to note that all values in the confidence interval are equally likely estimates of the true value of ($μ_1-μ_2$).

2. This blogpost, titled Within the Margin of Error, states:

   What I have in mind is misunderstanding about “margin of error” that treats all points within the confidence interval as equally likely, as if the central limit theorem implied a bounded uniform distribution instead of a t distribution. [...]
   The thing that talk about “margin of error” misses is that possibilities that are close to the point estimate are much more likely than possibilities that are at the edge of the margin".

These seem contradictory, so which is correct?

Answer 1

One question that needs to be answered is what does "likely" mean in this context?

If it means probability (as it is sometimes used as a synonym of) and we are using strict frequentist definitions then the true parameter value is a single value that does not change, so the probability (likelihood) of that point is 100% and all other values are 0%. So almost all are equally likely at 0%, but if the interval contains the true value, then it is different from the others.

If we use a Bayesian approach then the CI (Credible Interval) comes from the posterior distribution and you can compare the likelihood at the different points within the interval. Unless the posterior is perfectly uniform within the interval (theoretically possible I guess, but that would be a strange circumstance) then the values have different likelihoods.

If we use likely to be similar to confidence then think about it this way: Compute a 95% confidence interval, a 90% confidence interval, and an 85% confidence interval. We would be 5% confident that the true value lies in the region inside of the 95% interval but outside of the 90% interval, we could say that the true value is 5% likely to fall in that region. The same is true for the region that is inside the 90% interval but outside the 85% interval. So if every value is equally likely, then the size of the above 2 regions would need to be exactly the same and the same would hold true for the region inside a 10% confidence interval but outside a 5% confidence interval. None of the standard distributions that intervals are constructed using have this property (except special cases with 1 draw from a uniform).

You could further prove this to yourself by simulating a large number of datasets from known populations, computing the confidence interval of interest, then comparing how often the true parameter is closer to the point estimate than to each of the end points.

Answer 2

Suppose someone told me that I should place equal trust in all values within a CI95 as potential indicators of the population value. (I'm deliberately avoiding the terms "likely" and "probable.") What's special about 95? Nothing: to be consistent I would also have to place equal trust in all values within a CI96, a CI97, ... and a CI99.9999999. As the CI's coverage approached its limit, virtually all real numbers would have to be included. The preposterousness of this conclusion would lead me to reject the initial claim.

Answer 3

This is a great question! There is a mathematical concept called likelihood that will help you understand the issues. Fisher invented likelihood but considered it to be somewhat less desirable than probability, but likelihood turns out to be more 'primitive' than probability and Ian Hacking (1965) considered it to be axiomatic in that it is not provable. Likelihood underpins probability rather then the reverse.

Hacking, 1965. Logic of Statistical Inference.

Likelihood is not given the attention that it should have in standard textbooks of statistics, for no good reason. It differs from probability in having almost exactly the properties that one would expect, and likelihood functions and intervals are very useful for inference. Perhaps likelihood is not liked by some statisticians because there is sometimes no 'proper' way to derive the relevant likelihood functions. However, in many cases the likelihood functions are obvious and well defined. A study of likelihoods for inference should probably start with Richard Royall's small and easy to understand book called Statistical Evidence: a Likelihood Paradigm.

The answer to your question is that no, the points within any interval do not all have the same likelihood. Those at the edges of a confidence interval usually have lower likelihoods than others towards the centre of the interval. Of course, the conventional confidence interval tells you nothing directly about the parameter relevant to the particular experiment. Neyman's confidence intervals are 'global' in that they are designed to have long-run properties rather than 'local' properties relevant to the experiment in hand. (Happily good long-run performance can be interpreted in the local, but that is an intellectual shortcut rather than a mathematical reality.) Likelihood intervals—in the cases where they can be constructed—directly reflect the likelihood relating the experiment in hand. There is less about likelihood intervals that is confusing than is the case for confidence intervals, in my opinion, and they have more utility than might be expected from their almost complete disuse!

Answer 4

Let's start with the definition of a confidence interval. If I say that a 95% confidence interval goes from this to that I mean that statements of that nature will be true about 95% of the time and false about 5% of the time. I do not necessarily mean that I am 95% confident about this particular statement. A 90% confidence interval will be narrower and an 80% narrower still. Therefore, when wondering what the true value is, I have less credence in values as they get closer and closer to the edge of any particular confidence interval.

Note that all of the above is qualitative, especially "credence". (I avoided the term "confidence" or "likelihood" in that statement because they carry mathematical baggage that may differ from our intuitive baggage.) Bayesian approaches would rephrase your question to something that has a quantitative answer but I don't want to open that can of worms here.

Box, Hunter & Hunter's classic text ("Statistics for Experimenters", Wiley, 1978) may also help. See "Sets of Confidence Intervals" on pp 113, ff.