Confidence Intervals and Probability Relationship

Question

Suppose that 20 students visit a farmer's market and each pick (a random sample of) 25 oranges, weigh them, then create a 95% confidence interval for the true mean weight of an orange at the market. What is the probability that 5 of these intervals contain the true mean weight of an orange at the market and the rest don't?

According to Probability that multiple confidence intervals contain the true population mean, according to a comment, the answer should be ${20}\choose{5}$$(0.95)^5(0.05)^{15}$. However, according to the answer at the end, we can't just say that the probability of finding the true mean inside an interval is $1 - \alpha$.

I'm not sure which of these statements are right.

Answer 1

I think the confusion comes from how we interpret frequentist confidence intervals. There is NOT a 95% probability that the true mean lies in the interval. In a frequentist approach, the mean is fixed and not random. The interval is the random aspect. For a given interval, the true mean is either in the interval or not, there is no probability involved. However, the intervals are constructed in such a way that at least 95% of them contain the true mean.

That being said, I think it's fair to say if $\theta$ is our fixed true mean and $I(data)$ is a random 95% confidence interval (which depends on the data), then $P(\theta \in I) = 0.95$. Then the random variable "Exactly X out of 20 95% confidence intervals contain the true mean" can be viewed as a Binomial distribution with parameters (20, 0.95). Therefore $P(X = 5 | n=20, p=0.95) = \binom{20}{5}0.95^5 0.05^{15}$ as you originally stated.

Answer 2

Assumptions for my answer

Based on your description I understand that you are discussing a fixed population - oranges at one market at one point in time. You have twenty independent samplings, you do not specify if the oranges are replaced or if the samplings are mutally exclusive, nor what the total number of oranges are available.

For the sake of the answer I assume the total available was much larger than the number of oranges sampled, sufficicently larger that the distinction between replacing oranges (so that each student sampled the same population) as opposed to non-replacement (each subsequent student sampled an ever decreasing population) becomes negligible.

My Answer

In this case the 95% confidence interval is the orange within which you would expect to find the true mean in 95% of the experiments.

Caveats

If any of the assumptions degrade e.g.

1. non overlapping batch sampling combined with a low total number available (this makes the samplings interdependent due to non overlap constraints)
2. sampling by each student was carried out on different days with restocking , especially if seasons change between (population is constantly changing, each sampling within it is not randomly sampled across the whole distribution)
3. each student takes all their sampling from a single stall, but may sample different stalls (not randomly sampled across whole population)
4. if any student samples using a different criterion or chooses oranges non randomly, for example one chooses the ones they would want to eat which may in turn tend to be bigger (biased sampling will prevent the confidence interval from covering the true mean with the expected frequency)

Many more exclusions exist, under which the answer no longer applies as rigorously. In these cases it is not the definition that is the problem, it is invalid assumptions about sampling.

Did you follow the link provided by Néstor in the answer you referenced? It was What's the difference between a confidence interval and a credible interval? and explains when confidence intervals are relevant and when they are not - it depends precisely on what angle you need look at the data to answer your specific problem.