Consider the Pfizer vaccine for COVID-19. The data from the phase 3 trials shows that they had 21720 people in the vaccinated group and 21728 people in the control group:

n1 = 21720
n2 = 21728

In the vaccinated group, 8 people were infected with the virus. In the control group, 162 people were infected with the virus.

x1 = 8
x2 = 162

The ratios of infection are:

r1 = x1 / n1 = 0.000368
r2 = x2 / n2 = 0.007456

So, if the control group gives us the infection rate in the absence of the vaccine, how many of those infections were eliminated by the vaccine in the other group? The difference (that was eliminated) is r2 — r1, and we have to compare that with the baseline r2 to obtain the efficiency of the vaccine:

E = (r2 — r1) / r2 = 95.06%

Okay. But what is the 95% confidence interval for E? If you apply the procedure from this question...

Confidence interval around the ratio of two proportions

...then the CI95 you get is 89.96% ... 97.57%

But couldn't you use the Fisher exact test instead? (if no, why?). In R, this would be:

> vac = matrix(c(21720 - 8, 21728 - 162, 8, 162), nrow = 2)
> vac
      [,1] [,2]
[1,] 21712    8
[2,] 21566  162
> fisher.test(vac)

    Fisher's Exact Test for Count Data

data:  vac
p-value < 2.2e-16
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 10.09795 48.05249
sample estimates:
odds ratio 

But if you transform the CI for odds into percentages (100 * odds / (1 + odds)), you get 90.989% ... 97.961%. The interval has shifted a little, compared to the previous technique.

What am I missing?

  • 1
    $\begingroup$ "What am I missing" You are missing that there is nothing wrong with a different outcome. The alternative methods are approximations and it is expected to give not exactly the same result. The true distribution is a geometric distribution, and this can be approximated with a binomial distribution and this can be approximated with a normal distribution. $\endgroup$ – Sextus Empiricus Jan 27 at 14:34

The simplest way to find an approximate confidence interval for the odds ratio uses the delta method in the same was as the link you posted for the CI for the ratio of two proportions. The asymptotic standard error for the log-odds in one group with $x$ successes and $n$ patients with estimated probability of success equal to $\hat{p}=\frac{x}{n}$ is $\sqrt{\frac{1}{n \hat{p} (1-\hat{p})}}$. The estimated standard error for the log odds ratio is the square root of the sum of the two standard errors squared from the two groups $$\sqrt{\frac{1}{n_0 \hat{p_0} (1-\hat{p_0})}+\frac{1}{n_1 \hat{p_1} (1-\hat{p_1})}}$$

The R function fisher.test uses a more complicated algorithm to estimate both the odds ratio (not necessarily the naive estimate plugging in the estimates for the rates in the two arms) and the confidence interval. You can read the documentation for details. In this case, the CI for the odds ratio is not even symmetric around the point estimate (on the original or on the log scale). However, even using the simple approximation above, the CI will be symmetric on the log scale, but you cannot expect to be able to convert exactly from the CI for the odds ratio to the CI for the ratio of proportions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.