Equivalence of confidence intervals and hypothesis tests While reading John Rice's Mathematical Statistics and Data Analysis (3rd edition) and I came across this theorem in Chapter 9 (page 338):

Suppose that for every $\theta_0$ in $\Theta$ there is test at level $\alpha$ of the hypothesis $H_0:\theta = \theta_0$. Denote the acceptance region of the test $A(\theta_0)$. Then the set $$C({\bf X}) = \{\theta : {\bf X} \in A(\theta)\}$$ is a $100(1-\alpha)\%$ confidence region for $\theta$.

Can someone help me understand this statement? What does $C({\bf X})$ look like? An example would be very helpful.
 A: Let $X_1, X_2, \dots, X_9$ be a random sample from $\mathsf{Norm}{\mu, \sigma=15},$ with unknown mean $\mu.$ Suppose
$\bar X = \frac{1}{9} \sum_{i=1}^9 X_i = 92.20225$ be the sample mean,
as for the fictitious data sampled in R below.
set.seed(2022)
x = rnorm(9, 102, 15)
mean(x)
[1] 92.20225

Of course, in a real application, we would not know the true population mean.
Suppose we want to test $H_0: \mu=100$ against $H_a: \mu \ne 100$
at the 5% level.
Then the test statistic is $Z = \frac{\bar X - 100}{15/3} = -1.559549,$ as computed below.
z = (mean(x) - 100)/5;  z
[1] -1.559549

In terms of $Z,$ the acceptance region is $|Z| < 1.959964.$ So, we do not reject $H_0.$
A corresponding 95% confidence interval for $\mu$ is of the form
$$\bar X \pm 1.959964\frac{\sigma}{\sqrt{n}},$$ which
is $(82.40243, 102.00207).$
CI = mean(x) + qnorm(c(.025,.975))*(15/3);  CI
[1]  82.40243 102.00207
qnorm(c(.025,.975))
[1] -1.959964  1.959964

In terms of the test statistic $Z,$ the endpoints of this CI
are the endpoints of the 'acceptance' region.
(CI - mean(x))/5
[1] -1.959964  1.959964

In effect, on the scale of the data, the confidence interval consists of the 'acceptable'
values of $\bar X.$
A: This is the most general definition of a confidence interval. To understand why such a complicated definition is necessary, have a look at the definition usually found in introductory textbooks:
$$P_{cov}(\theta)=P(\theta \in [\theta_1,\theta_2]) = 1−\alpha$$
where $\theta_{1/2}$ are the limits of the confidence interval and are thus estimators to be computed from the observed data. Unfortunately, this equation contains the unknown true parameter value $\theta$ and thus cannot be solved for $\theta_{1/2}$, except for the special case that $\theta_{1/2}=\hat{\theta}\pm \delta$ and that $P_{cov}$ is a function of $\theta-\hat{\theta}$. This holds for the mean value of a normal distribution, which is the only case introductory text books cover and it is thus reasonable from a didactical point of view to use this (misleading, but easier to understand) definition.
It does not generalize, however, even to the simple case of a binomial proportion. Therefore, the problem that the true parameter value is unkown must be solved differently: if the true parameter lies on the border of the confidence interval, the probability of a deviation $|\theta-\hat{\theta}|$ greater than the observed value should be less than $\alpha$. This is equivalent to the limit of the acceptance region in hypothesis testing and leads to the definition in your post.
To actually compute a confidence interval, the definition in your question is still too general, because there are many ways to split the error probability $\alpha$. A definition that actually allows for a computation of $\theta_{1/2}$ distributes $\alpha$ equally to both sides:
$$P_{\theta=\theta_1}(\hat{\theta}\geq\theta_0) = \alpha/2 \quad\mbox{and}\quad
P_{\theta=\theta_2}(\hat{\theta}\leq\theta_0) =  \alpha/2$$
where $\theta_0$ is the observed value for $\hat{\theta}$.
