Double Blind Studies & Confounders

Question

I'm currently reading The Book of Why, and on pg 147-149 it talks about double blind RCT being very highly regarded in experiments, especially in the way it address confounders.

In theory, I can understand the confounders wash out in the limit of infinite trials. In practice, however, such as in the field of medicine, advertising, or crop yields in response to fertiliser choice as the book uses, I suspect there's rarely sufficiently many trials to even partially obtain the asymptotic properties of eliminating the effects of confounders. Without the ability to do a huge number of actual random trials, not actively controlling for confounders is worse, I would argue.

Moreover, in the case of double blind RCT, how does making potentially confounding factors unobservable to both the experimenter and the study participant improve study outcomes? One such example which come to mind is self-selection bias. Another is censorship in the context of survival analysis. Whether the study is double blind makes no difference to these, whereas they can be controlled for if the experimenter is allowed to active design their study.

So perhaps the double blind is simply there to address biases from the experimenter and participant, and to make studies more comparable across the board. But even this I take issue with because I think is equivalent to saying, on average, the ability for experimenters to control for confounders is worse than random. The only scenario where this makes sense is every study controlling for the same thing, which results in systematic bias. This is a fair concern, but I'm not sure how realistic it is. In all other cases, the individual experimental biases average out to noise in aggregate.

The other argument for double blind RCT is that it makes studies' results comparable, but I'd argue single blind is just as comparable. To me it seems obvious the residual effect after controlling for the major known factors will give stronger indication of the strength of the effect than those obtained which does not control for them. Eg. Suppose we have three studies on the effect of X on Y. The first controls for A, B, and C. The second study controls for A, B, D. The third controls for C, D, F. Then we have another set of double blind studies which controls for none of A-F. I just picture the mixture distribution of the first set of studies to be that much clearer than the second set due their attempt at isolating effect from noise.

Answer 1

You seem to be mixing a few things here. Unknown confounders are addressed thought randomization, because doing so properly guarantees (in the limit) that any characteristic that may be prognostic for your outcome is balanced across interventions.

Often randomization is stratified for one or a few of such traits that are known to correlate with the outcome, to make sure at least those occur with roughly equal probability in each group. The magic of letting chance decide who gets what is that any confounder -- even unknown! -- will eventually balance out (assuming your sample size is sufficiently large), and its effect will cancel itself in a comparison if it is not otherwise affected by only a specific intervention. Rare confounders will be more difficult to balance randomly, but their effects will be smaller on the average outcome anyway.

Blinding is there to address human bias, and has little to do with confounders. If you think a decision such as keeping a subject on treatment would be the same if a physician knew an investigative drug were active or not, you haven't worked with many investigators. That's not a criticism, they want what's best for their patients, but blinding prevents anyone from systematically steering towards a specific outcome (I really want this drug to work!). Essentially you need to act as if everyone receives the same unknown intervention.

This applies to more than the participant and the investigator in larger clinical studies by the way, and goes up to sponsor staff that's only involved in high-level strategic planning (by then that's usually quadruple-blind?). Pre-planned analysis approaches will be set in place before the blind is lifted. All of this is because you cannot disprove bias, you can only systematically exclude it through proper procedures. A conservative but not unrealistic assumption is that people planning and conducting the study will make decisions in such a way to give it the best chance of succeeding: no one likes failure. Maintaining the blind still means you have to address undesired/unexpected outcomes (adverse reactions, dropouts, deaths..), but you cannot systematically plan for them in such a way that favours any one intervention (we're going to count this death but not that one).

Neither blinding nor randomization guarantee comparability. That has to do with a lot of other factors as well, such as what population you're studying & under which conditions. Balancing out confounders through randomization and preventing bias via blinding will improve generalizability however, again because no one is able to act conditional on treatment allocation.

Answer 2

Adding to PBulls excellent answer:

You can still add covariates/confounders to the analysis in a double blind RCT. It's true that, with random assignment to treatment/control (or more) the confounders will balance out, on average (and statistical theory can make quite precise statements about how many subjects you need to get how much of a chance of getting a certain degree of balance), those other variables can still affect the regression (or whatever analysis you are doing).

The terminology does get confusing! There is random selection and random assignment and a "single blind" could be either the participant or the experimenter. And, in today's world, as PBulls points out "experimenter" is rarely one person. There can be a person collecting the data, a person analyzing the data, another one writing it up .... Indeed, more than one person doing each of these!

And these all relate to internal and external validity, which sometimes work at cross-purposes.