In Clinical trials - a methodologic perspective, Steven Piantadosi writes (ch.13, p. 334):

In Chapter 2, I noted the objections to randomization by Abel and Koch (1997) and Urbach (1993), and indicated the worth of studying their concerns and likely errors. They reject randomization as a

  1. means to validate certain statistical tests,
  2. basis for causal inference,
  3. facilitation of masking, and
  4. method to balance comparisons groups.

According to me, (1)-(4) are benefits of randomization. So, why do Abel, Koch, and Urbach reject randomization on the basis of those arguments?

  • 11
    $\begingroup$ I think you'd have to summarize the arguments that Abel and Koch and Urbach make, if you want people here to critique them. Otherwise, only people who have the book will be able to comment. The link you provided only shows things like programs used in the book $\endgroup$ – Peter Flom - Reinstate Monica Nov 27 '11 at 13:00
  • 4
    $\begingroup$ A more recent (2002) sympathetic summary of Urbach's (1993) arguments is available at What is Evidence? (stop-cocaine.co.uk/pdf/What%20is%20Evidence.pdf). $\endgroup$ – whuber Nov 27 '11 at 18:17
  • 1
    $\begingroup$ I have removed the hyperlink from my previous comment because that summary is no longer available and the resulting landing page is irrelevant. $\endgroup$ – whuber Jan 31 '16 at 14:31
  • 1
    $\begingroup$ Abstract of an Abel and Koch 1999 paper on the topic at least has the abstract available is at ncbi.nlm.nih.gov/pubmed/10408986. $\endgroup$ – Peter Ellis Jan 6 '17 at 3:27
  • $\begingroup$ An archived version of Urbach's paper is here: onlinelibrary.wiley.com/doi/10.1002/sim.4780121508/epdf. $\endgroup$ – AdamO Dec 5 '17 at 14:57

The papers from Koch, Abel, and Urbach do not reject randomization summarily as a means to achieve 1-4, rather they claim it is neither sufficient nor necessary to achieve those criteria. The take-home message is a) An RCT must not necessarily be done to answer every scientific question and b) Any published RCT may not be gold-standard evidence of efficacy.

As an alternative to the RCT, the open label trial (OLTs) is an obvious choice since the presumptive purpose of said trial is to evaluate a novel therapy not readily accessible by the patient population. For the analysis of RCT or OLT, similar principals to analyzing observational studies apply: control of causal factors, block randomization, and so on improve the efficiency and reduce the bias of such studies.

means to validate certain statistical tests

(are randomized participants "independent" and "identically distributed" per assumptions of t-test, log-rank test, and so on?)

RCT pros: Clusters of correlated participants are likely to be "broken up" in study randomization so that, without contamination, the dependence structure is similar within treatment assignment and methods for independent data estimate the correct standard errors anyway. Similarly, prognostic factors are likely to be balanced between study groups at the time of randomization.

RCT cons: Randomization does not address contamination: participants as a consequence of their indication and even participation in the study are likely to relate to one another and influence participation and outcomes as a result. Even with blocking, the distribution of prognostic factors is heterogeneous between arms. Those receiving the higher risk treatment and who are at higher risk at baseline are more likely to "die off" sooner, leading to a healthy risk set at future event times (survivor bias). This can lead to crossing hazards which is inefficient for log-rank tests.

basis for causal inference,

is the estimated effect the same as a "rewind-time" instance of assigning all treated participants to control, and subtracting those differences

RCT+: assignment of treatment is completely at random, no confounding by indication, blinding (when possible) may reduce risk of differential treatment discontinuation.

RCT-: Differential and non-differential follow-up due to attrition will contribute to imbalanced participants upon study completion. Non-blinded studies introduce risk of differential treatment discontinuation. Study parameters around randomization, blinding, and invasive therapies necessarily restrict the eligible study pool to a smaller subset which will consent to those parameters (healthy participant bias).

facilitation of masking:

when treatment is randomly assigned, is it possible to administer both treatments in a way that participants do not know what arm they have been randomized to?

RCT+: When an appropriate placebo is available, it can be done. It should be noted that the appropriate use of "placebo" is such that a participant receives standard of care (SOC). For instance, suppose an IND is administered by injection and SOC is a pill. Control participants receive SOC in an (unlabeled) pill form and a saline injection, while active arm participants receive the IND injection and an identical sugar pill.

RCT-: A placebo may not be available. For instance, provenge is a monoclonal antibody therapy for high grade prostate cancer. Administration of this treatment requires an invasive procedure called leukapheresis. Leukapheresis is too invasive and costly to ethically be performed in the control arm, so provenge-assigned participants will know they are receiving the IND.

method to balance comparisons groups.

is the expected distribution of "covariates" in the analysis sample equal in distribution between IND-treated and control participants?

RCT+: at time of randomization a 50/50 sample balance of treatment and control groups is noted, as well as an expected probabilistic balance of possible prognostic factors. Re-randomization is possible for batch-entry designs although they are far less prevalent these days.

RCT-: efficient design still requires control of prognostic factors, the optimal design in presence of a treatment effect is not 50/50 balance for most analyses, attrition and unequal cluster size due to loss-to-follow-up commonly means that balanced design is not guaranteed. Randomization does not guarantee balance of prognostic factors.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.