# Which binomial confidence interval is correct?

Background: I am working with a data set that requires a transformation. It's prevalence data so I have proportions to deal with, and as the proportions are quite low, I'm using the Freeman-Tukey transformation. My aim is to perform a meta analysis on the prevalence data.

I have transformed the proportions, and found confidence intervals using the transformed data.

I have a forest plot with CIs calculated exactly, and another with CIs calculated after a backtransformation. The largest difference between the two sets is 0.07, so they are very similar.

My issue is deciding whether I should be reporting the exact confidence intervals, or those that have been back transformed. There are ten studies in my data, so an approximation is not appropriate.

Question: In order to gain the correct confidence intervals, do I have to perform a back transformation?

I currently have two sets of answers and I'm not sure of the correct method.

Example: Let's say I have a proportion: 123/9876.

(1) In calculating the exact CIs without transformation, I get:

p=0.01245443; LB=0.01036126; UB=0.01484199


(2) After transforming the original data, and using (p-z*SE(p), p+z*SE(p)), where SE(p)=sqrt(1/(n+0.5)), I get:

p=0.224109; LB=0.2043868; UB=0.2438312


(3) Back transforming gives:

p=0.01245443; LB=0.01035768; UB=0.01474083


But which of these three results is correct?

• See this great answer: stats.stackexchange.com/questions/82720/… – Tim Mar 28 '17 at 9:49
• 123/4321 is not 0.01245443. – Wolfgang Mar 28 '17 at 11:06
• Apologies, I initially used an example with 4321 but decided I wanted to make the proportion smaller to make it clear that I was using the Freeman-Tukey transformation. I have edited it now. – Tom Mar 28 '17 at 11:52
• In my real example, I have a forest plot with CIs calculated exactly, and another with CIs calculated after a backtransformation. The largest difference between the two sets is 0.07, so they are very similar. My issue is deciding whether I should be reporting the exact confidence intervals, or those that have been back transformed. There are ten studies in my data, so an approximation is not appropriate. – Tom Mar 28 '17 at 11:55

The Wilson score interval is a simple and accurate confidence interval for the binomial proportion parameter, that automatically adjusts near the boundaries of the range. Suppose you observe $$N_1$$ "successes" and $$N_0$$ "failures" giving a total of $$n=N_0+N_1$$ data points. The Wilson score interval uses the normal approximation to give the following pivotal quantity:

$$\frac{(N_1 - n \theta)^2}{n\theta (1-\theta)} \overset{\text{Approx}}{\sim} \text{ChiSq}(1),$$

Letting $$\chi_{\alpha}^2$$ denote the critical point (upper tail area of $$\alpha$$) of the chi-squared distribution with one degree-of-freedom, and solving the resulting quadratic inequality for $$\theta$$, then gives the probability interval:

\begin{align} 1-\alpha &\approx \mathbb{P} \Bigg( (N_1 - n \theta)^2 \leqslant n \theta (1-\theta) \cdot \chi_{\alpha}^2 \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \theta \in \Bigg[ \frac{2N_1 + \chi_{\alpha}^2}{2n + \chi_{\alpha}^2} \pm \frac{n \chi_{\alpha}}{n + \chi_{\alpha}^2} \sqrt{\frac{N_0 N_1}{n} + \frac{\chi_{\alpha}^2}{4}} \Bigg] \Bigg). \\[6pt] \end{align}

Substitution of the observed values of $$n_0$$ and $$n_1$$ then gives the resulting confidence interval. This confidence interval is implemented in various functions. You can implement this interval in R using the binconf function from the Hmisc package. Here is the interval you get with your data:

Hmisc::binconf(x = 123, n = 9876, method = 'wilson');

PointEst      Lower      Upper
0.01245443 0.01044898 0.01483903


You need to report the back-transformed ones. This is because the original proportions are very small, you need to transform them and then transform them back into proportions.