What is a generalized confidence interval?

Question

Quoting Weerahandi, Generalized Confidence Intervals (1993):

• Confidence interval (Property 1) --- Consider a particular situation of interval estimation of a parameter $\theta$. If the same experiment is repeated a large number of times to obtain new sets of observations, then the $95\%$-confidence intervals will correctly include the true value of the parameter $\theta$ $95\%$ of the time.

• Generalized confidence interval (Property 2) --- After a large number of independent situations of setting (generalized) $95\%$-confidence intervals for certain parameters of interest, the investigator will have correctly included the true values of the parameters in the corresponding intervals $95\%$ of the time.

• Property 1 implies Property 2.

This definition of a generalized confidence interval is not clear to me. Is there a mathematical, rigorous phrasing of Property 2? This is my main question.

I'm also not able to figure out what this property means in rigorous terms with the help of the mathematical construction of generalized confidence intervals (using generalized pivotal quantities). How can we "see" Property 2 from this construction?

It is also not clear to me why Property 2 does not imply Property 1. If the same expriment is repeated, these are not two "independent situations" in the sense of Property 2?

Note: It is known that Property 2 does not imply Property 1; a GCI is not a CI in general.

---

Edit

Theorem 2.1 of this paper gives more information. Its statement is the following one.

For every integer $k \geq 1$, let ${\cal M}_k$ be a statistical model with unknown parameter $\theta_k$. The ${\cal M}_k$'s have independent sample spaces.

Assume there are some observations for each ${\cal M}_k$ and denote by $I_k$ a corresponding generalized $95\%$-confidence interval.

Set $\delta_k=1$ if $\theta_k \in I_k$ and $\bar\delta_n = \frac{\sum_{k=1}^n\delta_k}{n}$. Then $\Pr(\lim_{n\to\infty}\bar\delta_n = 95\%)=1$. That is, as $n\to\infty$, the mean number of the GCIs $I_k$ containing $\theta_k$ tends to $95\%$ almost surely.

Answer 1

The two properties imply each other. Indeed, the implication is nearly trivial provided we formulate them mathematically, as you have requested: let's begin there.

I would like to remark that the language is confusing because it is attempting to make statements about probabilities by referring to "a large number of": in other words, it is appealing implicitly to laws of large numbers to equate probabilities with asymptotic frequencies in sequences of independent trials. I will avoid such solecisms by translating these statements into what I think they are trying to say about probabilities.

For background, let us understand a "situation" to consist of collecting $n$ independent observations $\mathbf{x}=(x_1, x_2, \ldots, x_n)$ from some distribution $F$ assumed to lie within a specific family $\Omega$ of distributions.

A "parameter" is a function $\delta:\Omega\to\mathbb{R}$.

An "interval" can be represented as an indexed pair of functions $l_n,u_n:\mathbb{R}^n\to\mathbb{R}$ with the restriction that $l_n(\mathbf{x})\le u_n(\mathbf{x})$ for all $n\ge 1$ and all $\mathbf{x}\in\mathbb{R}^n$. For any $\mathbf{x}$, these functions determine the interval $[l_n(\mathbf{x}), u_n(\mathbf{x})]$.

1. A $1-\alpha$ confidence interval for $\delta$ is a pair $l_n, u_n$ for which $${\Pr}_F(\delta(F)\in [l_n(\mathbf{x}), u_n(\mathbf{x})])=1-\alpha$$ for all $F\in\Omega$ and all $n\ge 1$.

2. Consider a nonempty set of distributional families and parameters $\mathcal{S}=\{\delta_i:\Omega_i\to\mathbb{R}\}$. This models "a large number of independent situations." A $1-\alpha$ generalized confidence interval for $\mathcal{S}$ is a collection of functions $l_{i;n}, u_{i;n}$ indexed by $i\in\mathcal{S}$ and integers $n\ge 1$ such that for any sequence of length $s$ of samples $\mathbf{x}_j$ of sizes $n_j$ taken from $\Omega_j$, $$\frac{1}{s}\sum_{j=1}^s {\Pr}_F(\delta_{i_j}(\mathbf{x}_j)\in [l_{i_j;n_j}(\mathbf{x}_j), u_{i_j;n_j}(\mathbf{x}_j)])=1-\alpha.\tag{2}$$for all sequences $(F_j)$ with $F_j\in\Omega_j$.

By taking $\mathcal{S}=\{\delta\}$, $(2)$ trivially implies $(1)$. To go the other way, let $l_{i_j;n},u_{i_j;n}$ be a $1-\alpha$ confidence interval for each situation $j$: since all the probabilities appearing in formula $(2)$ exactly equal $1-\alpha$, the mean (on the left) also is $1-\alpha$.

Answer 2

under construction

This is very similar to Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

Your first property is more strict because it requires that the confidence interval contains 95% of the time the true parameter, conditional on the true parameter. The second property does not specify this conditional probability.

(The first property is not so explicit but you have this contrast between ”the same experiment is repeated a large number of times” and "After a large number of independent situations".)

This graph below comparing a confidence interval and a credible interval might be helpful to show the difference. In the left image we see that a credible interval will not be containing the true parameter exactly 95% of the time conditional on the particular true parameter value. For some parameters it will be more for others it will be less. But on average (averaging over the prior distribution of the parameter) it contains the parameter 95% of the time.