Why aren't these sentences about confidence intervals equivalent? Yes, I'm aware that there's similar/duplicated questions already open:

*

*Aren't these statements about confidence intervals equivalent?


*Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?


*Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals


*When do (and don't) confidence intervals and credible intervals coincide?
But funny enough, NONE of these questions has any answer marked as the right one, so this question is, technically, still unsolved in this site. I have read the answers and other related questions but I don't exactly see the difference yet so I will try to explain why I think these are equivalent expressions.
Given a confidence interval $CI_D$ for some unknown fixed parameter $\theta$ calculated from a dataset $D$, why aren't these three sentences equivalent?

*

*If I repeat the experiment an infinite amount of times and I get an infinite amount of calculated confidance intervals, 95% of them will contain $\theta$.

*There's a 95% probability that $CI_D$ contains $\theta$.

*There's a 95% probability that $\theta$ is in $CI_D$.

Why I think these sentences are equivalent?

*

*Sentences 1 and 2 are equivalent for me because I don't know if $CI_D$ is one of the 95% of intervals that contains $\theta$, which is the same as saying that $CI_D$ has a probability of 95% of being one of intervals that contains $\theta$, which is the same as saying that there's a probability of 95% that $CI_D$ contains $\theta$.


*Senteces 2 and 3 are equivalent because "$CI_D$ contains $\theta$" is equivalent to say "$\theta$ is in $CI_D$", because both sentences are translated as $\theta\in CI_D$. So $P[CI_D\ contains\ \theta] = 0.95$ is the same as saying $P[\theta\ is\ in\ CI_D] = 0.95$, because they are both the same as saying $P[\theta\in CI_D]=0.95$.
Am I right?
I know that frequentists doesn't allow to say "probability of fact around $X$" when $X$ is not a random variable (and $\theta$ is not because it's a constant), but $CI_D$ is and 2 and 3 speak about the probability of the relationship between $\theta$ and $CI_D$. So I'm not fully convinced that "probability of $\theta\in CI_D$" goes against the fact that $\theta$ is not a random variable, because $CI_D$ is a random variable and is also present in the same sentence (it tells something about $\theta$, but it also tells something about the $CI_D$).
 A: The events in statements 2 and 3 are obviously equivalent --- I interpret them as $CI_D \ni \theta$ and $\theta \in CI_D$ respectively.  The issue here is that you are vague about whether you are talking about CIs as random intervals or as fixed intervals after the observed data has been substituted, and you are also vague about whether you are talking about conditional or unconditional probability.  Below I will show which mathematical statements about confidence intervals are true/false.  So long as you describe these statements correctly in a textual sense (which requires more explicit specification of some issues you're glossing over) you should be fine.

Probabilistic properties of the CI: I'll conduct a purely probabilistic analysis of confidence intervals as mathematical objects, so I'll examine probability statements applying to these objects that are both conditional and unconditional on $\theta$.  Note that in the classical framework, the parameter is treated as an "unknown constant" so we (implicitly) condition on it in all probability statements in that context.  Nevertheless, I'll look at things more broadly so that you can see what probabilistic statements are true/false within a generalised framework where you examine the CI on a purely mathematical basis.
In order to show you what statements about confidence intervals are true/false, we will use more detailed notation.  Let $\text{CI}_\theta(\mathbf{X}, \alpha)$ denote the $1-\alpha$ level confidence interval for $\theta \in \Theta$ using (random) data vector $\mathbf{X}$.  This object is a mapping $\text{CI}_\theta: \mathbb{R}^n \times [0,1] \rightarrow \mathfrak{p}(\mathbb{R})$ that maps an input data vector and significance value to a measureable subset of the real numbers.  (For a confidence interval the output of the function is a single connected interval, but you can generalise to use confidence sets if you want to remove this restriction.)As I've n  oted in several other answers (some for questions you link to), an exact confidence interval is defined by the following property:
$$\mathbb{P}(\theta \in \text{CI}_\theta(\mathbf{X}, \alpha) | \theta) 
= 1-\alpha \quad \quad \quad \quad \text{for all } \theta \in \Theta.$$
(An approximate confidence interval is one where there is approximate equality, usually relying on asymptotic distributional results.)  Substituting the observed data $\mathbf{X}=\mathbf{x}$ then gives the (fixed) confidence interval $\text{CI}_\theta(\mathbf{x}, \alpha)$.  To allow us to assess statements about "repeated experiments" we will let $\mathbf{X}_1, \mathbf{X}_2, \mathbf{X}_3, ...$ denote a sequence of IID random vectors with distribution equivalent to the random vector $\mathbf{X}$.
So, assuming you are using an exact confidence interval, the following statements are true/false$^\dagger$:
$$\begin{align}
\mathbb{P}(\theta \in \text{CI}_\theta(\mathbf{X}, \alpha) | \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{True} \\[12pt]
\mathbb{P}(\text{CI}_\theta(\mathbf{X}, \alpha) \ni \theta | \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{True} \\[12pt]
\mathbb{P}(\theta \in \text{CI}_\theta(\mathbf{X}, \alpha)) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{True} \\[12pt]
\mathbb{P}(\text{CI}_\theta(\mathbf{X}, \alpha) \ni \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad  \quad \quad \quad \text{True} \\[12pt]
-------------&---------------- \\[6pt]
\mathbb{P}(\theta \in \text{CI}_\theta(\mathbf{x}, \alpha) | \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{False}^\dagger \\[12pt]
\mathbb{P}(\text{CI}_\theta(\mathbf{x}, \alpha) \ni \theta | \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{False}^\dagger \\[12pt]
\mathbb{P}(\theta \in \text{CI}_\theta(\mathbf{x}, \alpha)) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{False}^\dagger \\[12pt]
\mathbb{P}(\text{CI}_\theta(\mathbf{x}, \alpha) \ni \theta) 
&= 1-\alpha \quad \quad \quad \quad \quad \quad \quad \quad \text{False}^\dagger \\[12pt]
-------------&---------------- \\[6pt]
\mathbb{P} \bigg( \lim_{k \rightarrow \infty} \frac{1}{k} \sum_{i=1}^k \mathbb{I}(\theta \in \text{CI}_\theta(\mathbf{X}_i, \alpha)) 
&= 1-\alpha \bigg| \theta \bigg) = 1 \quad \quad \quad \quad \ \ \text{True} \\[6pt]
\mathbb{P} \bigg( \lim_{k \rightarrow \infty} \frac{1}{k} \sum_{i=1}^k \mathbb{I}(\theta \in \text{CI}_\theta(\mathbf{X}_i, \alpha)) 
&= 1-\alpha \bigg) = 1 \quad \quad \quad \quad \quad \ \text{True} \\[6pt]
\end{align}$$
If you are working in the classical ("frequentist") context, you can ignore the marginal probability statements here and focus entirely on the conditional probability statements.  (In that context the parameter is an "unknown constant" and so all our probabilistic analysis implicitly conditions on it having a fixed value.)  As you can see, the remaining distinction that determines whether the statement is true/false is whether you are talking about the "data" in its random sense or fixed sense.  You also need to take care to state these mathematical conditions clearly and accurately.

$^\dagger$ Statements listed as $\text{False}$ are statements that are not true in general.  These statements may be true "coincidentally" for some specific values of the inputs.
A: Sentence 1 is the de jure interpretation of confidence intervals. I like to say it as follows:

The 95% in "95% confidence interval" refers to the long term relative frequency of these estimators containing the true estimand upon repeated construction under ideal and identical circumstances.

Before embarking, I'd like to point out that $ \theta \in CI_D$ can be interpreted as either $\theta$ being in $CI_D$ as in sentence 3, or $CI_D$ covers $\theta$ as in sentence 2.  Hence, sentences 2 and 3 are (from where I stand) equivalent.  Let's discuss why sentence 1 and 2 (or 1 and 3) are thus not equivalent.
The statement made in sentence 2 seems to imply that $\theta$ is the random quantity. Frequentists treat estimands as fixed.  Hence sentences 1 and 2 are not equivalent merely based on this fact.  Now, people will often defend interpreting confidence intervals in the style of sentence 2 (or 3) by referencing the estimators long term relative frequency, however I think this is an error.  Any given confidence interval either contains the estimand (assuming it is fixed) or it does not.  There is no probabalistic element to this, unless we were to repeat the construction of the interval with different data (which would appeal to the definition). Its like asking "what is the probability that the card on the top of this deck is an ace?".  The card is either an ace or it isn't, there is no randomness here.  When people answer "the probability is 4/52" what they really mean is "In an infinite sequence, were I to come to well shuffled decks then 4/52 decks would have an ace on top".  These are two very different scenarios.
That's my argument, let's take a look at some of yours:

Sentences 1 and 2 are equivalent for me because I don't know if CIDCI_D is one of the 95% of intervals that contains θ\theta, which is the same as saying that CIDCI_D has a probability of 95% of being one of intervals that contains θ\theta, which is the same as saying that there's a probability of 95% that CIDCI_D contains θ.

Here, I think you are making the mistake I spoke of earlier using the decks.  The interval is fixed, the parameter is fixed, the interval either covers it or it does not.  This means the probability is either 0 or 1, but we don't know.  All we know is the long term relative frequency of intervals capturing the estimated under ideal conditions, and we just hope that this happens to be one of those times where our interval covers the estimand.

Senteces 2 and 3 are equivalent because "$CI_D$ contains $\theta$" is equivalent to say "$\theta$ is in $CI_D$", because both sentences are translated as $\theta\in CI_D$. So $P[CI_D\ contains\ \theta] = 0.95$ is the same as saying $P[\theta\ is\ in\ CI_D] = 0.95$, because they are both the same as saying $P[\theta\in CI_D]=0.95$.

I disagree with the statement $P[CI_D\ contains\ \theta] = 0.95$, because as noted before, any given interval either covers the estimand or not. The probabalistic statement is about infinite sequences of intervals.
If you're interested, I wrote a little about this here on my blog.
A: The key thing to know is that the frequentist confidence interval say nothing directly about the unknown fixed value of $\theta$, the local parameter of interest for the experiment that you ran. Instead, the frequentist confidence interval is the result of a method that yields intervals with on average the nominal coverage (with a few small caveats) when applied in the long run to analyses of experiments for all possible values of $\theta$.
Some types of frequentist confidence interval that yield exactly the nominal coverage for all possible values of the parameter of interest (e.g. the ordinary Student's t confidence interval for the random sample-based estimate of the mean of a normally distributed population). If your interval is of that type then you can indirectly infer something about the probability of your fixed unknown instance of the parameter falling within the interval. But that probability is not a 'proper' frequentist probability because that 'proper' probability is either one or zero. Of course, even that indirect inference is only well-formed if you know for certain that your data are obtained in circumstances that allow the distributional and sampling assumptions of the method to be correct.
Other types of frequentist interval do not have uniformly nominal coverage. Cases where the data are discrete provide useful examples. See here, for example: Discrete functions: Confidence interval coverage? If your fixed unknown value of $\theta$ lies in an 'unlucky' region of parameter space then the coverage of the method can be remarkably far from nominal.
Where the coverage is not exactly nominal over all possible values of the parameter you cannot assume that the coverage is nominal for your fixed unknown value of the parameter and so you cannot make any exact statement about the probability that you value lies inside the interval.
If you need to know the probability that an interval contains the unknown value of $\theta$ then you need to use a method that allows direct specification of probabilities of non-random parameters. In other words you need Bayesian probabilities.
It is true that very often the Baysian credible intervals are very similar to (or identical to) some frequentist intervals (as long as the priors allow) and so Bayesian statements about the presence of fixed parameter values within frequentist intervals are not too misleading even if they are incorrect.
