Can the Fisher exact test be used to get a confidence interval for the efficiency of a vaccine (risk ratio or relative risk)?

Question

Consider the Pfizer vaccine for COVID-19. The data from the phase 3 trials shows that they had 21720 people in the vaccinated group and 21728 people in the control group:

n1 = 21720
n2 = 21728

In the vaccinated group, 8 people were infected with the virus. In the control group, 162 people were infected with the virus.

x1 = 8
x2 = 162

The ratios of infection are:

r1 = x1 / n1 = 0.000368
r2 = x2 / n2 = 0.007456

So, if the control group gives us the infection rate in the absence of the vaccine, how many of those infections were eliminated by the vaccine in the other group? The difference (that was eliminated) is r2 — r1, and we have to compare that with the baseline r2 to obtain the efficiency of the vaccine:

E = (r2 — r1) / r2 = 95.06%

Okay. But what is the 95% confidence interval for E? If you apply the procedure from this question...

Confidence interval around the ratio of two proportions

...then the CI95 you get is 89.96% ... 97.57%

But couldn't you use the Fisher exact test instead? (if no, why?). In R, this would be:

> vac = matrix(c(21720 - 8, 21728 - 162, 8, 162), nrow = 2)
> vac
      [,1] [,2]
[1,] 21712    8
[2,] 21566  162
> fisher.test(vac)

    Fisher's Exact Test for Count Data

data:  vac
p-value < 2.2e-16
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 10.09795 48.05249
sample estimates:
odds ratio 
  20.38808

But if you transform the CI for odds into percentages (100 * odds / (1 + odds)), you get 90.989% ... 97.961%. The interval has shifted a little, compared to the previous technique.

What am I missing?

Answer 1

The simplest way to find an approximate confidence interval for the odds ratio uses the delta method in the same was as the link you posted for the CI for the ratio of two proportions. The asymptotic standard error for the log-odds in one group with $x$ successes and $n$ patients with estimated probability of success equal to $\hat{p}=\frac{x}{n}$ is $\sqrt{\frac{1}{n \hat{p} (1-\hat{p})}}$. The estimated standard error for the log odds ratio is the square root of the sum of the two standard errors squared from the two groups $$\sqrt{\frac{1}{n_0 \hat{p_0} (1-\hat{p_0})}+\frac{1}{n_1 \hat{p_1} (1-\hat{p_1})}}$$

The R function fisher.test uses a more complicated algorithm to estimate both the odds ratio (not necessarily the naive estimate plugging in the estimates for the rates in the two arms) and the confidence interval. You can read the documentation for details. In this case, the CI for the odds ratio is not even symmetric around the point estimate (on the original or on the log scale). However, even using the simple approximation above, the CI will be symmetric on the log scale, but you cannot expect to be able to convert exactly from the CI for the odds ratio to the CI for the ratio of proportions.