# What is a generalized confidence interval?

Quoting Weerahandi, Generalized Confidence Intervals (1993):

• Confidence interval (Property 1) --- Consider a particular situation of interval estimation of a parameter $\theta$. If the same experiment is repeated a large number of times to obtain new sets of observations, then the $95\%$-confidence intervals will correctly include the true value of the parameter $\theta$ $95\%$ of the time.

• Generalized confidence interval (Property 2) --- After a large number of independent situations of setting (generalized) $95\%$-confidence intervals for certain parameters of interest, the investigator will have correctly included the true values of the parameters in the corresponding intervals $95\%$ of the time.

• Property 1 implies Property 2.

This definition of a generalized confidence interval is not clear to me. Is there a mathematical, rigorous phrasing of Property 2? This is my main question.

I'm also not able to figure out what this property means in rigorous terms with the help of the mathematical construction of generalized confidence intervals (using generalized pivotal quantities). How can we "see" Property 2 from this construction?

It is also not clear to me why Property 2 does not imply Property 1. If the same expriment is repeated, these are not two "independent situations" in the sense of Property 2?

Note: It is known that Property 2 does not imply Property 1; a GCI is not a CI in general.

### Edit

Theorem 2.1 of this paper gives more information. Its statement is the following one.

For every integer $k \geq 1$, let ${\cal M}_k$ be a statistical model with unknown parameter $\theta_k$. The ${\cal M}_k$'s have independent sample spaces.

Assume there are some observations for each ${\cal M}_k$ and denote by $I_k$ a corresponding generalized $95\%$-confidence interval.

Set $\delta_k=1$ if $\theta_k \in I_k$ and $\bar\delta_n = \frac{\sum_{k=1}^n\delta_k}{n}$. Then $\Pr(\lim_{n\to\infty}\bar\delta_n = 95\%)=1$. That is, as $n\to\infty$, the mean number of the GCIs $I_k$ containing $\theta_k$ tends to $95\%$ almost surely.

• When I Google "generalized confidence interval", this question is the first hit, suggesting that the concept has not caught on, maybe because nobody has been able to understand how it differs from the traditional confidence interval concept. Commented Nov 27, 2020 at 11:14
– Ben
Commented Nov 28, 2020 at 0:30
• @Ben he does in the beginning Commented Jul 22, 2021 at 1:25
• @Taylor: Okay, thanks. Perhaps OP could add page numbers to the citations so that it is simple to find the relevant definitions/theorems.
– Ben
Commented Jul 22, 2021 at 7:01

The two properties imply each other. Indeed, the implication is nearly trivial provided we formulate them mathematically, as you have requested: let's begin there.

I would like to remark that the language is confusing because it is attempting to make statements about probabilities by referring to "a large number of": in other words, it is appealing implicitly to laws of large numbers to equate probabilities with asymptotic frequencies in sequences of independent trials. I will avoid such solecisms by translating these statements into what I think they are trying to say about probabilities.

For background, let us understand a "situation" to consist of collecting $n$ independent observations $\mathbf{x}=(x_1, x_2, \ldots, x_n)$ from some distribution $F$ assumed to lie within a specific family $\Omega$ of distributions.

A "parameter" is a function $\delta:\Omega\to\mathbb{R}$.

An "interval" can be represented as an indexed pair of functions $l_n,u_n:\mathbb{R}^n\to\mathbb{R}$ with the restriction that $l_n(\mathbf{x})\le u_n(\mathbf{x})$ for all $n\ge 1$ and all $\mathbf{x}\in\mathbb{R}^n$. For any $\mathbf{x}$, these functions determine the interval $[l_n(\mathbf{x}), u_n(\mathbf{x})]$.

1. A $1-\alpha$ confidence interval for $\delta$ is a pair $l_n, u_n$ for which $${\Pr}_F(\delta(F)\in [l_n(\mathbf{x}), u_n(\mathbf{x})])=1-\alpha$$ for all $F\in\Omega$ and all $n\ge 1$.

2. Consider a nonempty set of distributional families and parameters $\mathcal{S}=\{\delta_i:\Omega_i\to\mathbb{R}\}$. This models "a large number of independent situations." A $1-\alpha$ generalized confidence interval for $\mathcal{S}$ is a collection of functions $l_{i;n}, u_{i;n}$ indexed by $i\in\mathcal{S}$ and integers $n\ge 1$ such that for any sequence of length $s$ of samples $\mathbf{x}_j$ of sizes $n_j$ taken from $\Omega_j$, $$\frac{1}{s}\sum_{j=1}^s {\Pr}_F(\delta_{i_j}(\mathbf{x}_j)\in [l_{i_j;n_j}(\mathbf{x}_j), u_{i_j;n_j}(\mathbf{x}_j)])=1-\alpha.\tag{2}$$for all sequences $(F_j)$ with $F_j\in\Omega_j$.

By taking $\mathcal{S}=\{\delta\}$, $(2)$ trivially implies $(1)$. To go the other way, let $l_{i_j;n},u_{i_j;n}$ be a $1-\alpha$ confidence interval for each situation $j$: since all the probabilities appearing in formula $(2)$ exactly equal $1-\alpha$, the mean (on the left) also is $1-\alpha$.

• Hello whuber. It is not true that a generalized confidence interval is a confidence interval. Hence Property 2 should not imply Property 1. I should have mentioned this point in my post. Commented May 14, 2017 at 18:17
• Thank you. In that case, your quotation does not provide a sufficiently clear definition. Could you provide any information to make it less ambiguous?
– whuber
Commented May 14, 2017 at 22:05
• Hi whuber. I can provide the definition of the construction of a GCI, but I'm not sure this will help. I'll do it later. Commented May 26, 2017 at 10:56
• Although a construction could help illustrate the GCI, what we need is a sufficiently clear definition.
– whuber
Commented May 26, 2017 at 12:23
• Sure. But if I had a clear definition I would not have opened this question ^^ Commented May 26, 2017 at 13:02

under construction

This is very similar to Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

Your first property is more strict because it requires that the confidence interval contains 95% of the time the true parameter, conditional on the true parameter. The second property does not specify this conditional probability.

(The first property is not so explicit but you have this contrast between ”the same experiment is repeated a large number of times” and "After a large number of independent situations".)

This graph below comparing a confidence interval and a credible interval might be helpful to show the difference. In the left image we see that a credible interval will not be containing the true parameter exactly 95% of the time conditional on the particular true parameter value. For some parameters it will be more for others it will be less. But on average (averaging over the prior distribution of the parameter) it contains the parameter 95% of the time.

• Sorry I don't get your point. Why are you introducing the Bayesian stuff? This has nothing to do with the question. Commented Nov 27, 2020 at 12:39
• @StéphaneLaurent I would disagree that it has not anything to do with it. I believe that a Bayesian credible interval is an example of a generalized confidence interval, because it satisfies the condition from the theorem (if the prior is correct). A credible interval will not contain the parameter 95% of the time for a specific $\theta_k$ (So it is not a confidence interval). But for all $\theta_k$ it will on average be correct 95% of the time. Commented Nov 27, 2020 at 12:51
• That is not true. An obvious counter-example is when you take a prior distribution whose support does not contain the true value of the parameter. Then the posterior interval never catches the true value. Commented Nov 27, 2020 at 12:56
• @StéphaneLaurent of course the model (like the prior) needs to be correct. The same is true for confidence intervals, they will also be wrong if the model (e.g. the likelihood function) is incorrect. Commented Nov 27, 2020 at 12:59
• And what is a correct prior? And do you have some references to support your claims? Commented Nov 27, 2020 at 13:00