I understand that randomised controlled trials (RCTs) are used to perform causal inference, but I'm a confused about how this is reasonable. Let's say that we have a treatment, and we want to find out if this treatment "works". We randomly allocate participants into two groups, and one group is allocated the treatment and the other the placebo/control. But these people are not clones of each other operating in identical, controlled environments. And my understanding of RCTs is that, often, these people go about their normal lives whilst undertaking the trial – they're not isolated into some totally controlled, hermetic environment. So we have people who are vastly different, who then operate in vastly different, uncontrolled environments during the trial. How can this possibly reasonably allow us to perform causal inference? How does randomly allocating participants into two groups and randomly allocating them to treatment/control make up for all of these innumerable other possible factors that could influence variables of interest? How do we know whether any effects, either in one direction or another, were caused by the treatment, or some other factor (of which, as I said, there are innumerable)? It kind of seems to me like trying to find a needle in a haystack in a hurricane – not reasonable/realistic.
-
$\begingroup$ Clearly you would not be convinced by an RCT which just had two participants (one treated and one control) even with the randomisation, for just the reasons you give. But the hope is that with larger groups of subjects, the impacts of other differences between participants will be averaged down to being small enough that the effects of the treatment (if any) will become visible. The randomisation avoids biases between groups among these other differences. $\endgroup$– HenryCommented Feb 9 at 23:42
-
5$\begingroup$ Not just "hoping" but quantifying the amount of hope by accounting for the variations in people (or other subjects). That's what inferential statistics is all about. $\endgroup$– Peter FlomCommented Feb 9 at 23:56
-
1$\begingroup$ That's how "proofs" work with statistics -- there is in fact no "proof" or "guarantee" that something works or doesn't work. You get it right if the estimate of the uncertainty is correct. $\endgroup$– dipetkovCommented Feb 10 at 0:48
-
2$\begingroup$ Stephen Senn has written a lot on this topic. See for example Randomisation is not about balance, nor about homogeneity but about randomness. There is more in the section on randomization. Balance is what you refer to as "totally controlled, hermetic environment". $\endgroup$– dipetkovCommented Feb 10 at 0:56
-
2$\begingroup$ @ThePointer No, my comment is in agreement with David's answer. We don't have to think about the other factors (although it may be useful to do so) but we can still quantify the accuracy. This gets attributed to error. If there is a lot of error, then you won't get significance $\endgroup$– Peter FlomCommented Feb 10 at 12:42
1 Answer
The randomization of participants into different groups, eg, treatment and control. is central to the inference of causality about the treatment. The randomization makes the treatment groups closely similar, on average. Larger randomized groups are more similar.
High variation among the participants is essential to concluding that any treatment will work in the larger population, with its high environmental, social, and even genetic variation. Designers of trials make a serious effort to include human variation in their study populations. Randomization makes the two groups as similar as possible in the presence of wide individual variation.
There is a chance, of course, as with any experiment, that the appearance of a real difference between treatment and control with a significant statistical test is wrong. This chance is reflected in the p-value. If this is small then the chance of a false result is small. This supports the conclusion that the treatment caused the difference.
Many clinical trials are done on groups of people with one strong similarity, the presence of a disease, such as a specific diagnosis of cancer. Otherwise, participants often reflect the variety found among humans. That variety is necessary to conclude any effect can be relied on in other people with the same disease.
A large clinical trial is usually planned after a treatment is found to be reasonable safe and promising for a small number of people.
If a study is well designed with adequate power and does not yield a significant difference then this is usually thought to mean the effect of the treatment is small or nonexistent,certainly not as large as originally thought.
-
$\begingroup$ So, in RCTs, we are not trying to explicitly account for significant factors, for instance, as one would in regression analysis by selecting them as predictors, but rather to (implicitly) bundle all significant (and insignificant) factors other than the treatment(s) together into a kind of "high variance factor", and then analysing the treatments against that "high variance factor"? So, that way, we don't need to go through all of the trouble of trying to think of, and account for, all of the possible significant factors that could influence our variable of interest? $\endgroup$ Commented Feb 10 at 0:42
-
$\begingroup$ Yes. I'm not quite sure what you mean by a "high variance factor". Some factors that affect the outcome might be used as covariates in analysis or as categories during design, by randomizing in each category. Most such sources of variation are not identifiable, that is, residual error in regression. And yes, ignoring them makes design easier. Thinking about them is always desirable and might give a better design. $\endgroup$ Commented Feb 10 at 0:50
-
$\begingroup$ For instance, unless we are keeping all subjects isolated in identically controlled environments for the duration of our experiment, it might be totally reasonable that there are innumerable other factors, other than the treatments, that are significantly affecting the variable of interest. These other factors could be, for instance, diet, exercise, sleep, environment, psychological state, etc. It is not feasible to explicitly account for all of these other significant factors, even though they may all significantly affect the variable of interest. [...] $\endgroup$ Commented Feb 10 at 0:57
-
$\begingroup$ [...] So, in RCTs, do they just not account for all such variables explicitly, and instead account for them implicitly by "bundling" them into a single "high variance factor", similar to the error term in regressions? I mean, these might all be significant factors, but we cannot reasonably account for them in our RCT, right? So how are RCTs accurate, or at all useful, for performing causal inference, since such critical/significant factors cannot be accounted for? That's going to the core of my argument/question. $\endgroup$ Commented Feb 10 at 1:01
-
1$\begingroup$ Yes. It is both reasonable and valid and gives causal inference. You seem to think that a myriad of other possible causal factors are known and measurable. They aren't. $\endgroup$ Commented Feb 10 at 3:25