Bias of difference-in-means estimator for experiments randomized using Bernoulli trials

Question

Under potential outcomes framework (Neyman-Rubin causal model), it is straighforward to show that difference-in-means is an unbiased estimator of average treatment effect under completely randomized trials (when we fix the number of experiment units in treatment/control groups).

However, in Bernoulli trials experiment, each unit is randomly assigned to treatment/control group with some fixed probability. The bias computation of difference-in-means estimator is more difficult because the number of units in control/treatment group is now a random variable.

Is the difference-in-means still unbiased for Bernoulli trials randomized experiments? Is the way to prove this using law of iterated expectation, by conditioning on the number of units in treatment group?

Answer 1

The unbiasedness of the difference in means estimator for the average treatment effect is a direct result of uncofnoundedness and causal consistency, and otherwise is unrelated to the treatment assignment mechanism (i.e., whether it is a Bernoulli trial or a completely randomized trial).

$$E[Y|A = a] =E[Y(a)|A=a]=E[Y(a)]$$ for each treatment vaue $a$, implying the difference in means estimator, which is unbiased for $E[Y|A = 1] - E[Y|A = 0]$, is unbiased for $E[Y(1)] - E[Y(0)]$.

The first equality is due to causal consistency, i.e., $Y = AY(1) + (1-A)Y(0)$, which implies that $Y(a)$ and $Y$ are equivalent when conditoning on $A=a$. The second equality comes from confoundedness, that $A \perp Y(a)$ for each $a$, which implies that $f(Y(a)|A) = f(Y(a))$. In a Bernoulli trial, unconfoundedness holds because assignment to treatment does not depend on any factors related to the potential outcomes (i.e., it is a fixed probability for each individual).

Answer 2

With Bernoulli randomization, there is a small chance we end up with all units being randomized to the same group. When that happens, the difference-in-means estimator is undefined.

In the event that we have some units in both groups, the difference-in-means estimator is defined and is unbiased. This can be seen using iterated expectation first conditioning on the random vector of assigned treatments, and then marginalizing over it.