Linear regression and causality in a randomized controlled experiment

Question

We know that a linear regression Y on X doesn't imply causation X->Y. It just means that Y is dependant on X in this model. For example, in a general case, I cannot simply run a regression of Test score on class size and conclude that class size is a cause of test score.

Now, assuming we have a randomized controlled experiment. This means:

• all subjects ideally follow the protocol
• there is a controlled group and a treatment group.
• subjects are randomly assigned
• the experiment is measuring if a treatment has an effect

Then, we run a regression for example test score on class size.

Can we say that class size is a cause of test score ?

And what do you think of this sentence ?

A causal effect is defined to be the effect measured in an ideal randomized controlled experiment.

Answer 1

It comes down to how well your experiment isolates the hypothesized causal factor. As you mention, the general procedure is to randomly assign subjects to control or treatment groups. The purpose of doing this is to be able to make an all-else-equal comparison between groups. Ideally, the factor of interest (e.g. class size in your example) is the only thing that differs between the two groups.

There are various circumstances where this can break down. For example, imagine you're testing a drug to see whether it's effective. You randomly assign patients to the control or treatment groups. The treatment group receives the drug. Say the treatment group has a better outcome than the control group. Can we say the drug caused the caused the improved outcome? Not necessarily. For example, say the control group received nothing. It could be that the act of receiving treatment caused the improved outcome, but not the drug itself. This is why the control group in such studies must receive a placebo (fake treatment). Furthermore, they must not know that they're in the control group, because simply knowing whether you're receiving a treatment or not can affect the outcome. This why blind studies (where subject don't know which group they're in) are important. It's also import to conduct such studies in a double-blind fashion, where the experimenters are also ignorant about whether they're administering a treatment or placebo. Otherwise, they might be biased to treat patients differently (for example, they might treat patients in the treatment group better than those in the control group, because they have a vested interest in the experiment having a positive result).

Of course, it's not always possible to conduct placebo-controlled, double-blind experiments. In the classroom example, there's no way that anybody could be ignorant of the class size. Placebos and blinding are just a couple examples of a more general principle: that causal inference can be thrown off when other causal factors 'leak' into the experiment (meaning that the control and treatment groups have differences other than the factor of interest, and these differences affect the outcome). The way to combat this is to design good controls.

Answer 2

The ability make to make a causal inference depends on the conditional independence of the potential outcomes and the treatment, to speak in counterfactual terms (e.g., Rosenbaum & Rubin, 1983). If potential outcomes are independent of treatment after accounting for the relevant covariates, then you can make a causal inference from the treatment to the outcome.

More simply, this is the idea of no unaccounted-for confounders. If all confounders are accounted for either in statistics or design, then there is no plausible alternative explanation for the observed effect than for the treatment to have caused it. So the reason randomization allows for a causal inference is that it, in theory, produces two balance groups that have no pre-existing differences in their background covariates. If there were pre-existing differences in their background covariates, and the outcomes of the two gorups were compared with a simple t-test, a possible alternative explanation is that the difference in the background covariates between the two groups is the true cause of the observed effect.

Using regression can allow for the estimation of a causal effect if after conditioning on the variables on the regression, there is still an effect, and there are no other variables that could plausibly confound the observed relationship conditional on the included covariates.

Other methods like matching attempt to mimic a randomized experiment in the sense that the two groups are balanced on all relevant covariates prior to effect estimation. Groups can be balanced by design (i.e., with randomization or matching), or by statistics (i.e., with regression or structural models).