# How exactly is the "effectiveness" in the Moderna and Pfizer vaccine trials estimated?

As in the title. Is this "a risk ratio"? How is it calculated, if you could provide an example with numbers for both trials, please? I am not a statistician, but I am familiar with the binomial distribution - I suppose it is used here to calculate the confidence intervals?

### Moderna

Based on the press release we can assume that there were 30 000 patients total and observed were 90 infections among placebo and 5 infections among the vaccinated group.

Let's assume that the vaccine group and placebo group were each of the same size 15 000.

So, calculated on the back of an envelope, instead of 90 infections you got 5 infections. The reduction due to the vaccine is 85 out of 90 patients that did not get infected (without vaccine 90 get infected with vaccine 5 get infected, so presumably the vaccine reduced it from 90 down to 5). This is the $$85/90 \approx 94.4 \%$$ that is the number you see in the news.

• This would normally need to be adjusted. The groups may not have been the same sizes and the people may not have been exposed at the same time (you do not get everybody vaccinated at exactly the same time). So eventually you will be doing some more complicated computation of the risk and based on the ratio of those figures you get to a more exact figure (but the on the back of an envelope calculation will be reasonably close).

• In addition, the $$94.4\%$$ is just a point estimate. Normally a range of confidence is given for an estimate (confidence interval). Roughly speaking this is a measure for how accurate/certain the measurement/estimate is. It gives some boundaries for failure of the estimate (typical are 95% boundaries).

One way to compute the confidence interval for ratio's is to express it in terms of log odds apply an approximation formula for the error use that to express the interval and then convert back to ratio's. This would give a $$95\%$$ confidence interval between $$88.0\%$$ and $$97.8\%$$ for the effectiveness.

$$\begin{array}{} \text{log_odds} &=& \log \frac{5}{90} \approx -2.89\\ \text{S.E.}_\text{log_odds} &\approx& \sqrt{\frac{1}{5}+\frac{1}{90}+\frac{1}{14995}+\frac{1}{14910}} \approx 0.460\\ CI_{95\%}(\text{log_odds}) &\approx& \text{log_odds}-1.96\text{S.E.}_\text{log_odds} \, , \, \text{log_odds}+1.96\text{S.E.}_\text{log_odds}\\ & \approx &-3.79,-1.99 \\ CI_{95\%}(\text{odds}) &\approx& 0.0225,\ 0.137 \\ CI_{95\%}(\text{effectivity}) &=& \frac{1}{1+CI_{95\%}(\text{odds})} \\&\approx& 88.0 \%,\ 97.8 \% \end{array}$$

These computations assume ideal situations (as if the numbers 5 and 90 stem from nicely understood causes for the variations). The assumption is not an interference that breaks the statistical model. E.g. patients that got vaccinated and had fever or other symptoms afterward may have been distancing more because of that. For them, the exposure is less and that is not taken into account in the on the back of the envelope calculation. In addition, this relates to effectivity for the total period (in which the infection pressure may not have been equally distributed). Based on these simple figures, we can not say with the same accuracy how effective the vaccination is as a function of time (especially the question whether the immunity decreases over time).

• Thank you very much for so detailed and easy answer. Now it's all clear to me! Commented Nov 17, 2020 at 0:55
• I believe it was 95 infections in total, see investors.modernatx.com/news-releases/news-release-details/…. Efficacy seems to be computed as 1-5/90, which - not that the difference matters quantitatively - also produces the published 94.5%, see skranz.github.io/r/2020/11/10/CovidVaccine.html. Commented Nov 17, 2020 at 9:00
• Anyone can refer to a paper or study? Have they made anything available? Commented Nov 18, 2020 at 6:53
• Pfizer has their validation protocol (including the statistical background) posted on their website Direct Link Commented Nov 18, 2020 at 13:05
• @Ashok the efficacy is a number that expresses how the vaccine works in comparison to a situation without a vaccine. In order to be able to compute such a number, you need to know what happens in both situations and therefore you have two groups in the study. You are right it can be done in multiple other ways, but this way (a randomized controlled trial) will have the least bias. Commented Mar 4, 2021 at 14:41