Which binomial confidence interval is correct?

Question

Background: I am working with a data set that requires a transformation. It's prevalence data so I have proportions to deal with, and as the proportions are quite low, I'm using the Freeman-Tukey transformation. My aim is to perform a meta analysis on the prevalence data.

I have transformed the proportions, and found confidence intervals using the transformed data.

I have a forest plot with CIs calculated exactly, and another with CIs calculated after a backtransformation. The largest difference between the two sets is 0.07, so they are very similar.

My issue is deciding whether I should be reporting the exact confidence intervals, or those that have been back transformed. There are ten studies in my data, so an approximation is not appropriate.

Question: In order to gain the correct confidence intervals, do I have to perform a back transformation?

I currently have two sets of answers and I'm not sure of the correct method.

Example: Let's say I have a proportion: 123/9876.

(1) In calculating the exact CIs without transformation, I get:

p=0.01245443; LB=0.01036126; UB=0.01484199

(2) After transforming the original data, and using (p-z*SE(p), p+z*SE(p)), where SE(p)=sqrt(1/(n+0.5)), I get:

p=0.224109; LB=0.2043868; UB=0.2438312

(3) Back transforming gives:

p=0.01245443; LB=0.01035768; UB=0.01474083

But which of these three results is correct?

Answer 1

The Wilson score interval is a simple and accurate confidence interval for the binomial proportion parameter, that automatically adjusts near the boundaries of the range. The coverage properties of various intervals have been examined in Brown, Cai and DasGupta (2001) and this is one of the intervals they recommend as having good coverage properties.

The interval is constructed as follows. Suppose you observe $N_1$ "successes" and $N_0$ "failures" giving a total of $n=N_0+N_1$ data points. The Wilson score interval uses the normal approximation to give the following pivotal quantity:

$$\frac{(N_1 - n \theta)^2}{n\theta (1-\theta)} \overset{\text{Approx}}{\sim} \text{ChiSq}(1),$$

Letting $\chi_{\alpha}^2$ denote the critical point (upper tail area of $\alpha$) of the chi-squared distribution with one degree-of-freedom, and solving the resulting quadratic inequality for $\theta$, then gives the probability interval:

$$\begin{align} 1-\alpha &\approx \mathbb{P} \Bigg( (N_1 - n \theta)^2 \leqslant n \theta (1-\theta) \cdot \chi_{\alpha}^2 \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \theta \in \Bigg[ \frac{2N_1 + \chi_{\alpha}^2}{2n + \chi_{\alpha}^2} \pm \frac{n \chi_{\alpha}}{n + \chi_{\alpha}^2} \sqrt{\frac{N_0 N_1}{n} + \frac{\chi_{\alpha}^2}{4}} \Bigg] \Bigg). \\[6pt] \end{align}$$

Substitution of the observed values of $n_0$ and $n_1$ then gives the resulting confidence interval. This confidence interval is implemented in various functions. You can implement this interval in R using the CONF.prop function from the stat.extend package. Here is the interval you get with your data:

#Compute confidence interval using Wilson-score method
library(stat.extend)
CONF.prop(alpha = 0.05, sample.prop = 123/9876, n = 9876)

Confidence Interval (CI) 
 
95.00% CI for proportion parameter for infinite population 
Interval uses 9876 binary data points with sample proportion = 0.0125 

[0.0104489771546811, 0.0148390256803883]

Answer 2

You need to report the back-transformed ones. This is because the original proportions are very small, you need to transform them and then transform them back into proportions.