What's wrong with this interpretation of a 95% confidence interval? Note: I asked a version of this as part of another question, but I'm re-asking it as a stand-alone question with more detail.
I've been trying to come up with more intuitive/less confusing ways to accurately explain what a given (say) 95% confidence interval (e.g. "the 95% CI runs between 2.5 and 3.7") actually means. I know that it does not mean "there is a 95% chance that the true value is within this interval" (because the true value is not a random variable) but that it is correct to say that, "out of all possible 95% CIs that could have been calculated from these data 95% of them will include the true value." However, based on my interpretation of the Central Limit Theorem and frequentism, I believe the following statement should also be valid interpretation of a 95% confidence interval that ranges between A and B:
"if it were true that the true mean was the the value we actually estimated, and we replicated our study a 100 times, estimating the mean each time, then 95% of those estimates would fall between A and B."
I've been told this is wrong, but I just want to understand why it's wrong.
Here's my logic. Let me know if and where there is a problem.
(First off, to avoid having to talk about t distributions let's just say that all sample sizes in this discussion are large enough that the distinction between normal and t distributions is irrelevant).
We want to estimate some parameter $\mu$ in a given population. The CTL says that if we draw a large number of random samples from this population and generate an estimate $\hat \mu$ in each of them, these estimates will form a normal curve, with a particular standard deviation, centered on the true value  $\mu$. We therefore know, for example, that 95 percent of these estimates will be within 1.96 standard deviations of the true value. Of course, we only have one of these estimates, and we don't know how big the standard deviation of this sampling distribution actually is. However, we use the standard deviation $\hat \sigma$  of  $\hat \mu$ in the data we actually collected as an unbiased estimator and thus estimate the standard error as $\hat \sigma / \sqrt n$.
Now, let's assume for the sake of argument that $ \mu=\hat \mu$. (obviously this is unlikely to be the case, but it is analogous to the hypothetical we use when interpreting p values: "if the null hypothesis were true..."). Under this assumption the CLT tells us that if we again took repeated random samples, other estimates of $ \mu$ from those samples will be distributed normally around $\hat \mu$  (because $ \mu=\hat \mu$), meaning that 95% of those estimates will be within 1.96 standard deviations of $\hat \mu$. As noted above, we estimate the size of "one standard deviation" in the sampling distribution by the standard error. This allows us to can calculate the following
$$A=\hat \mu+1.96*SE$$
$$B=\hat \mu-1.96*SE$$
We can therefore say "if $\mu= \hat \mu$, and we replicated our study a large number of times times, estimating the $\hat \mu$ each time, then 95% of those estimates of would fall between A and B."
Of course, the formula I used was just the standard formula to calculate a 95% confidence interval around $\hat \mu$. So it seems like the sentence above should be a valid interpretation of a 95% confidence interval. If it's not, why not?
 A: Your statement is imprecise in terms of true and estimated values, but not so much of the mean, but of the standard deviation.

if it were true that the true mean was the the value we actually estimated, and we replicated our study a 100 times, estimating the mean each time, then 95% of those estimates would fall between A and B.

Note that A and B depend on a single estimate of the standard deviation, namely the one from the original experiment. Your replications only estimate the mean, not standard deviations.
If your original experiment underestimated the SD, then replications will yield estimates of the mean that are contained in your original interval [A,B] in fewer than 95% of cases. If your original experiment overestimated the SD, your replicates will fall in the interval more often than 95%. And you will either under- or overestimate the SD with probability 1.
With whuber, the solution lies in keeping the CI calculation and a result from such a calculation (which is known as "a CI") strictly separate. A CI can only be interpreted as a particular result from a CI calculation. Yes, this is unintuitive. Unfortunately, there is simply no intuitive and correct interpretation of CIs.
A: The big problem with your statement ["if it were true that the true mean was the value we actually estimated, and we replicated our study a 100 times, estimating the mean each time, then 95% of those estimates would fall between A and B."] is that we want to make an inference about the true mean, whereas you are discussing a hypothetical application in which the true mean has a different value.
Regarding confidence intervals, to calculate a confidence interval you have already assumed a statistical model, say with random variable X, observed sample x, and random confidence intervals C(X) = [L(X, U(X)]. In short, the lower and upper limits bounce around from sample to sample but 95% of the possible intervals contain the true mean. Post-sample, my confidence interval C(x) = [L(x), U(x)] either does or does not contain the true mean. The genius of Neyman was to use the word “confidence” to explicitly acknowledge a different meaning than would obtain were the word “probability” used in place of it.
Suppose I compute a 95% confidence interval for the true mean. Given the assumed model, we state 95% confidence that the CI produced by my sample contains the true mean only because the assumed process (i.e., the model and C(X)) produces correct confidence intervals 95% of the time.
Sure, I don’t know whether the observed CI is correct or not, but the 95% accuracy of the process that generated it is somewhat reassuring. Given the assumed model, either the observed CI contains the true mean or we have suffered a rare event [Note the past tense here.] In particular, our 95% confidence claim is neither a probability, nor a posterior probability, nor a belief. Note that there is nothing random left, the observed confidence interval either contains the true mean or it does not. By using the word confidence, we face up to that reality.
Had the degree of confidence been higher, say 99.9%, we would have been even more confident (because if it doesn’t contain the true mean then it means I have suffered extreme bad luck [note past tense again] that occurs only 0.1% of the time).
There is an important caveat regarding such confidence claims. Confidence would be undermined if it was known that the confidence interval procedure that I was using had poor conditional properties. (There are a number of contrived examples that demonstrate this.) In such circumstances, it is better to base confidence claims on appropriate conditional probabilities rather than on the unconditional probability.
If you want an authority for why confidences are not the same as probabilities, the text Cox and Hinkley (1974) p. 227-228 spells out why confidences cannot be manipulated as if they are probabilities.
