For example, are more novel methodologies for obtaining unbiased treatment effect estimates, like Targeted Maximum Likelihood Estimation, being utilized in randomized controlled experiments in clinical trials for new drugs or vaccines?

Or due to small sample sizes and government regulations (depending on the country in question) are statisticians limited to more straightforward and explainable analysis methodologies (regression, exact matching) after designing the experiments?

  • 1
    $\begingroup$ For most randomized trials, the FDA sets the methodology, which is COXph. For ATE estimation in randomized trials, modern causal inference methods are not used as the primary analysis. However, it is being utilized for other analyses' in these trials. For instance, numerous targeted learning methods are being used to assess correlates of risk and protection for the COVID-19 vaccines using the FDA trial data. This recent manuscript for instance and the related SAP in references : medrxiv.org/content/10.1101/2021.08.09.21261290.abstract. $\endgroup$
    – user327671
    Commented Sep 1, 2021 at 6:41

1 Answer 1


If you design your clinical trial well (randomly assigning people either a treatment or a control condition) then there probably isn't any good reason to use any fancy statistical methods for obtaining unbiased treatment effects. The whole reason randomized assignment is the "gold standard" is that (if done correctly) it eliminates ANY possibility of bias due to confounding factors. So all you need to do is use a statistical test (like a t test or chi square test) account for statistical uncertainty associated with the fact that you only ran your trial on a sample of people, rather than the whole population.

Methods like regression analysis were developed for situations in which randomization isn't possible or doesn't work correctly. They are often considered a "second best option" to use when you can't use an ideal design. In reality, of course, randomization can fail, and you might be interested in differential treatment effects for certain groups, or other more complex questions. So there might be specific reasons to use modeling in those cases. But in a plane vanilla clinical trial where the randomization worked and you are only interested in the overall treatment effect I'm struggling to think of a good reason why you would want to go beyond a simple t test.

  • 4
    $\begingroup$ Though regression isn't necessary for RCTs, adjusting for pre-exposure variables related to the outcome can introduce additional efficient by reducing residual variability. SO regression in RCTs is not an "in case something went wrong" approach $\endgroup$ Commented Aug 31, 2021 at 18:42
  • 2
    $\begingroup$ Even when a randomization plan works perfectly, there can be some imbalance between two treatment groups. Many things can go wrong, eg, dropouts that might be due to a known or unknown risk factor for the disease. Clinical trials often use risk factors as covariates or use analyses suitable to outcomes over long times where the results of treatment are not final, eg, Cox models. The FDA decides if design and execution of a clinical trial is satisfactory and has generally, not always, required randomized trials since 1962. That was when new legislation required that drugs be proven effective. $\endgroup$ Commented Aug 31, 2021 at 19:34
  • $\begingroup$ Thanks Graham! I'm looking at a job description for a statistician position for a "data science and statistics team" (two separate fields?) with a biotech company for conducting clinical trials. The impression I'm getting is the analysis duties will be pretty tedious: t-tests and chi-squared tests, something in all honesty a company doesn't need a devoted statistician position to perform. $\endgroup$
    – RobertF
    Commented Sep 1, 2021 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.