3
$\begingroup$

On this page one can find the following statement:

Magnitude of Effect: Vaccination against HPV-16 and HPV-18 reduces incident and persistent infections with efficacy of 91.6% (95% confidence interval [CI], 64.5%–98.0%) and 100% (95% CI, 45%–100%), respectively. Efficacy beyond 6 to 8 years is not known.

Now, what I cannot understand is how can something with the 95% confidence interval of 45%-100% be claimed to be 100% efficacious? Is this some sort of medical jargon I am lacking?

| cite | improve this question | | | | |
$\endgroup$
  • 1
    $\begingroup$ I believe the trial(s) evaluated found 0 cases of infection (100% efficacy), and the CI indicates the degree of uncertainty. Without knowing how it was calculated, it's difficult to interpret. It may have been obtained from, e.g., an estimate of variance from prior trials. $\endgroup$ – djlid Jun 5 '19 at 23:16
  • $\begingroup$ @djlid Yes, I agree. Of course, the 100% is used by propagandists, even though it is completely meaningless. $\endgroup$ – Igor Rivin Jun 5 '19 at 23:17
1
$\begingroup$

Vaccine efficacy has a well-defined technical meaning, as explained on the Wikipedia page:

Vaccine efficacy is the percentage reduction of disease in a vaccinated group of people compared to an unvaccinated group, using the most favorable conditions.

One useful formula for vaccine efficacy (VE, in fractional instead of percentage terms), quoting from that Wikipedia page, is:

VE = 1 - RR

where RR is the relative risk of developing the disease for vaccinated people compared to unvaccinated people.

The statement that you quote is a summary (I suspect, a meta-analysis) of several studies described in detail farther down on that web page. Let's look at details of one of those studies, as summarized on the US CDC web page that you linked:

An international, double-blind, placebo-controlled trial of a bivalent HPV-16/HPV-18 VLP vaccine was performed in 1,113 women aged 15 to 25 years with normal cervical cytology who were seronegative for HPV-16, HPV-18, and 12 other oncogenic HPV types at enrollment. Women received either vaccine or placebo at 0, 1, and 6 months and were assessed by cervical cytology and self-obtained cervicovaginal samples for at least 18 months. A masked treatment-allocation follow-up study was performed for an additional 3 years, for a combined analysis of up to 6.4 years of follow-up. The 12-month persistent infection rate of HPV-16 or HPV-18 in an according-to-protocol cohort (i.e., women who received all three doses of vaccine or placebo on the correct schedule) was 0 of 401 women in the vaccine arm compared with 20 of 372 women in the placebo arm, with a vaccine efficacy of 100% (95% CI, 81.8–100).

Standard practice for any type of study is to report a point estimate of the statistic of interest and associated confidence intervals. There were 0 persistent HPV infections among the 401 women who received all 3 doses of vaccine versus 20/372 (5.4%) in those receiving 3 placebo doses. The observed risk ratio was 0 for vaccinated versus placebo.

Under the accepted definition of vaccine efficacy, the point estimate of efficacy for persistent infections in that study was thus 100%. Vaccine was associated with NO persistent infection in this study group. A vaccine efficacy of 100% is not, as you say in a comment, "completely meaningless"; that efficacy is what the results of that study showed. To report any other point estimate would, frankly, be a lie under the accepted definition.

Confidence intervals must cover the point estimate. So your statement in a comment on another answer that "the confidence interval is open on the right" is incorrect. If the point estimate is 100% efficacy, then the upper confidence interval limit must be closed at 100% to include the observed point estimate.

With respect to the lower confidence limit, there are several ways to estimate confidence intervals for relative risk (and thus for vaccine efficacy), as described on this page. These methods take into account the probability of finding by chance the observed numbers of persistent infections in vaccinated and placebo individuals.

The confidence-interval calculation methods can differ in cases like this where there was no infection in vaccinated cases. However they are calculated, they provide the following "confidence": if you repeat the experiment a very large number of times and calculate the confidence intervals according to the specified method, then in 95% of experiments the confidence interval will include the true value. So yes, it is possible that the true vaccine efficacy is less than 100%; reporting the confidence limits is a principled way of expressing the level of "confidence" in the point estimate.

There are certainly limitations of vaccine efficacy as defined above. Efficacy is defined as "using the most favorable conditions," those who met all study requirements. In this particularly study, only 774 (401+373) women out of 1113 entered into the study were included in efficacy analysis. The other 339 presumably did not receive all 3 vaccine or placebo doses or were not available for full follow-up. Vaccine effectiveness in contrast is how well the vaccine protects in practice. As might be expected, HPV vaccine effectiveness is lower, about 82% in one study that is cited farther down on the CDC web page you linked.

These values are not the output of "propagandists," as you claim in a comment on your question. They represent standard accepted statistical practice consistent with defined meanings of technical terms. Working with accepted definitions of technical terms provides a principled way to compare and combine information from multiple studies, as the CDC page you link attempts to do.

Strikingly, the overall point estimate of 100% efficacy of vaccination with respect to persistent infection means that none of the studies reviewed by the CDC found persistent HPV infections (infections than can lead to development of cancer) among vaccinated women. That provides very strong evidence that HPV vaccinations are efficacious even if they cannot be proved with finite sample sizes to provide perfect protection. That fact is particularly important as other studies cited on the CDC web page showed no significant difference in side effects between vaccine and placebo group. There is essentially no reason to forgo HPV vaccination, as it has clear benefits without statistical evidence of associated side effects.

| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ The "propagandists" comment is actually consistent with what you say: the confidence interval is the "principled" thing to report. The point estimate is of technical interest (translates absolute numbers into percentages, which is obviously useful), but not useful for reporting (after all, I can say that wearing Air Jordans is 100% efficacious in preventing alien abductions, but is this really useful?) Otherwise, thanks for the very informative answer! $\endgroup$ – Igor Rivin Jun 6 '19 at 4:41
  • $\begingroup$ As for the confidence intervals covering the point estimate. I cannot imagine any treatment for any condition being 100% effective, so I cannot see how 100% can be in the confidence interval. $\endgroup$ – Igor Rivin Jun 6 '19 at 4:43
1
$\begingroup$

I will propose a slightly different way of interpreting the 95% confidence interval in this context. As the sample resulted in an efficacy of 100%, we might ask what efficacy %s in the population would have 100% in the 95% range of "usual" values (for samples of the same size). Working this backward, we find that a population efficacy as low as 45% would still have samples with 100% efficacy showing up in the range of usual values. But, if we move just a bit below, say 44%, then a sample with 100% efficacy would be considered unusual.

| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ Yes, that is my interpretation exactly. Of course, an actual 100% is not possible, unless the vaccine kills the patient (who would then have complete immunity to HPV), so the confidence interval is open on the right. $\endgroup$ – Igor Rivin Jun 6 '19 at 0:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.