I am interested in statistical procedures and study designs used to determine the absence/acceptability of long-term side effects of covid vaccines. Given that these vaccines were created and approved within a short period of time (less than a year), it seems that the studies using conventional survival analysis are of little value. I assume that one relies on the similarity with other, well-tested vaccines.

I have posted earlier a similar question in the biology community, and some suggested that the question has to do more with statistics (and medicine) than with biology.

After a medication/vaccine has been developed and tested in a lab, it has to pass a clinical study, which is essentially a series of statistical studies aiming to demonstrate that the medication is safe, produces desired effect, and does not have serious side effects. the data from these studies are tested against the threshold established by FDA (or equivalent authorities in other countries), and only then the medication is allowed for use on general public.

Note that the approval thresholds can vary depending on the nature of medication: e.g., cancer treatments have well-known side effects (think of the chemotherapy), which are tolerated for the sake of saving human lives. On the other had, vaccines used on healthy public should not have any significant side effects. The vaccines against SARS-CoV-2 might fall somewhere in between, due to the urgency of the current health crisis.

Phase II of clinical trial is most known in the popular culture - it is at this phase that the participants are separated into the test and the control groups, administered respectively the real medication and the placebo, which rises variosu moral dilemmas. Phase II is preceeded by a similarly designed Phase I, where the medication is tested on healthy people, to test its toxicity. Phase III is a finer study of the dozage, the side effects, etc., which is the subject of thre question.

Phase IV of the clinical studies begins after the approval and commercialization of the medication. My understanding is that Phase IV is essentially an epidemiological study, designed as a survival analysis for incidence of various adverse health conditions (or death), with the use of medication/vaccination appearing among the covariates. If this covariate is found to be (statistically) significant in terms of the incidence of the adverse side effects, the medicatuon/vaccine may be prohibited for further used/withdrawn from the market.

Contrary to a widespread belief, vaccines are not by themselves "safe" or "dangerous". Rather their safety is determined on a case-by-case basis, by the amount of the adverse effects they produce. Thus, the most commonly used vaccines, those that might be obligatory in some countries (e.g., 11 vaccines are obligatory in France) have been typically in use for decades, and judged safe on the basis of the wealth of the available data. However, even a safe vaccine can be judged too risky, if the risk of contracting the disease is very low - e.g., the use of the smallpox vaccine has been discontinued, since this disease is considered eradicated in most of the world, although it still exist in labs and isolated cases are occasionally reported. Another example is the tuberculosis vaccine, which carries a more substantial risk of side effects, and therefore used only on the contact cases or when traveling to risky regions. Among the vaccines that failed Phase IV trials and had to be withdrawn is Dengvaxia by Sanofi Pasteur.

A related problem is that of long-term vaccine efficacy - flu vaccine is known to be notoriously inefficient, due to the high variability of the influenza virus, whereas the tetanus vaccine has to be re-administered every few years.

Let us now get back to SARS-CoV-2 (covid, coronavirus). There is no doubt that it is not poisonous and that it is efficient against the virus. However their long-term side effects obviously could not be tested via Phase IV survival analysis methodology. On the one hand, the risk thresholds for these vaccines are lower, due to the worldwide health emergency (which is why they were authorized via emergency authorization, even before being vetted via peer review process as scientific publications). I so believe that the Phase III clinical trials include statistical procedures/safeguards that allow to assure the lack of serious side effects in the foreseeable future. These statistical methods/designs are the point of my question.

  • 2
    $\begingroup$ It is not really about statistics but about medical research. For statistics you need data. Obviously, there is not long-term effects data for the vaccines, so statistics won't help. I guess you could try to extrapolate the results from other vaccines etc, but this would be more about using domain knowledge than statistics. $\endgroup$
    – Tim
    Commented Dec 29, 2020 at 20:37
  • $\begingroup$ @Tim the question is about the statistical methods for extrapolating the data from other vaccines. But yes, it is about statistical procedures specific to medicine. $\endgroup$
    – Roger V.
    Commented Dec 30, 2020 at 8:22
  • 2
    $\begingroup$ @Vadim "extrapolating the data from other vaccines" that means there's no data from covid-19 vaccins. This should be clarified and the question is more like 'what are long-term side effects from vaccins?'. This is still not really statistics. Sure, you can have some data about this and that can be treated statistically (one could make some prediction for the probability of a long-term effect). But the core of the question is medicine and statistics hardly plays a role. $\endgroup$ Commented Dec 30, 2020 at 10:59
  • $\begingroup$ @SextusEmpiricus Perhaps, the title is misleading. The question is about the clinical studies methodology, which is basically applied statistics... to the extent that one can consider biostatistics as statistics. $\endgroup$
    – Roger V.
    Commented Dec 30, 2020 at 11:12
  • $\begingroup$ I believe that the idea about long-term vaccin safety a theoretical concept and not something based on experimental tests. You have this stuff like a Hoskin's effect or antibody-depenedent enhancement and who knows there is more but it is not based on direct evidence/data.... $\endgroup$ Commented Dec 31, 2020 at 0:33

1 Answer 1


The FDA has not approved any covid vaccine, rather Pfizer, J&J, and AstraZeneca have been granted emergency use authorization. People receiving these vaccines not as study participants are basically joining a type of Phase IV long-term safety study. Within the ongoing phase 3 studies, Pfizer, J&J, and AZ are required to perform pharmacovigilance. The studies still continue as of my answering this question. See here: https://clinicaltrials.gov/ct2/show/NCT04368728.

Among the stated objectives, the Pfizer study (as an example) intends to report on the proportion of vaccinated patients experiencing "serious adverse events from dose 1 through 6 months after the last dose". This "through 6 months after the last dose" is considered the treatment-emergent window. SAEs after this interval continue to be collected and reported to the government and data-monitoring committees (DMC). Only the DMC has access to unblinded data. Each of these independent bodies has the right to halt, modify, or terminate the study on the basis of safety, conduct, or ethical concern.

A serious adverse event is any untoward experience leading to death, disability, congenital disorder, requires or prolongs hospitalization, or is considered life threatening, or other medically important condition. The nefarious Guillain-Barre syndrome in other vaccines is an example of just such a clinical event. There is also a case series of immune thrombocytopenia in the current SARS-COV2 vaccine candidates that is currently being followed https://www.nytimes.com/2021/02/08/health/immune-thrombocytopenia-covid-vaccine-blood.html

Short-term efficacy is an issue separate from long-term safety. Supposing a vaccine is safe but suffers from short-term efficacy, one could conceivably crush a pandemic through a mass vaccination campaign and eradicate the disease. We need to give serious consideration to the possible flaws of the staggered roll-out of vaccine rather than embarking on a mass vaccination campaign. If the disease is denied any host through herd immunity, it would conceivably be eradicated in a short time. However, as cases continue to rise among non-vaccinated individuals, we increase the risk of encountering a mutant strain that is not targeted by the previous vaccine.

Statistically, the methods to detect safety issues are not complicated. Indeed, time-to-event like Cochrane Mantel Haenszel or Cox analyses are used to compare rates of clinical events over time. Markers for possible emerging safety particularly platelets but also bilirubin, liver enzyme, creatinine, QT intervals, TNF-$\alpha$, and quality of life measures are also collected and monitored by-arm in the DMC to possible detect early safety concerns.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.