I want to analyze the results of a block randomized experiment. Within each block, units are randomized to treatment and control as if that block were a completely randomized experiment. Specifically, I want an unbiased estimator for the overall treatment effect and an estimate of the variance of this estimator.
If the probability of receiving treatment is the same in each block, then an unbiased estimate of the overall average treatment effect ATE can be obtained like so:
$$\hat{ATE} = \sum_{j=1}^J \frac{N_j}{N}\hat{ATE_j}$$ Where:
- N = Total Number of units in the experiment
- $N_j$ = Number of units in block j
- $\hat{ATE_j}$ is the Average Treatment Effect in block j. Specifically, $$ \hat{ATE_j} = \bar{Y}_{j,treatment} - \bar{Y}_{j,control}$$
$\bar{Y}_{j,treatment}$ is the average observed outcome in the treatment group in block j and $\bar{Y}_{j,control}$ is the average observed outcome in the control group in block j.
The difficulty is that the probability of treatment varies by the strata/blocks. So, some blocks have 50% of their units treated and others have, for example, 10% treated. I've been told that the above estimator for ATE is only unbiased if the probability of treatment is constant across the blocks and that, otherwise, I need to use some kind of Horvitz-Thompson weighting, weighting the units by their inverse probability of treatment, in order to get an unbiased estimator.
I can't find any literature on this. Do I need to use horvitz-thompson or some other kind of weighting? Or can I still use the estimator above? Any references would be especially appreciated.
Thanks!