4
$\begingroup$

When sample sizes are too small to trust that the usual asymptotics will guarantee good balance across experimental groups on known confounders, a common approach is stratified or blocked randomization. What, if any, are the practical or statistical downsides to such conditionally randomized approaches?

The only downside I can think of is that as the number of conditionally randomized factors increases, it becomes increasingly hard to actually implement the randomization scheme (with the limiting case being 1-1 matching, a definite headache-producer). Are there others?

$\endgroup$
4
$\begingroup$

This is a good question, and this might not be a good answer. However, it was too long for a comment and I felt like typing.

I would point you to this article, which states two major downsides to stratified randomization:

  1. Population must be classified into only 1 subpopulation each
  2. Sorting in grey areas is inefficient and brings in measurement bias.

I would also like to add that, at least from my knowledge, statistically you can never go wrong with stratification in non-trivial cases. It is easily proven that (assuming equal costs across $L$ strata such that $\sum^L_{i=1} C_i = L\cdot C^*$ and that $N$ is large such that $\frac{1}{N} \approx \frac{1}{N_i} \overset{.}{=} 0$)

$$ V_{opt} (\bar{y}_{st}) \leq V_{prop} (\bar{y}_{st}) \leq V_{SRS} (\bar{y}_{SRS}) $$

That is, you have the minimum (of these three) variance in your effect ($\bar{y}$) if you optimally assign (Neyman allocation) strata, followed by proportionally allocating samples in strata just based on their $N_i$ or number of population in that strata. Both of these are better than just simple random sampling, which is no stratification. So, from my view (and please prove me wrong), there is always a gain in accuracy when you have stratified sampling, and above this if $\bar{Y}_i$ is very different than $\bar{Y}$. That is, if the strata mean is very different from the grand mean.

Stratified sampling is better than simple random sampling by a factor of:

$$ \begin{aligned} V_{SRS} (\bar{y}) - V(\bar{y}_{st}) &= \frac{1}{n} \sum W_h (S_h - \bar{S})^2 + \frac{1-f}{n} \sum^L_{h=1} W_h (\bar{Y_h} - \bar{Y})^2 \end{aligned} $$

and Optimal allocation (incorporating costs) is better than simple random sampling and proportional allocation by a factor of:

$$ \begin{aligned} V_{ran} - V_{prop} &= \frac{1-f}{n} \sum^L_{h=1} W_h S^2_h + \frac{1-f}{n} \sum^L_{h=1} W_h (\bar{Y}_h - \bar{Y})^2 - \frac{1}{n} W_h S^2_h + \frac{1}{N} W_h S^2_h \\ &= \frac{1-f}{n} \sum^L_{h=1} W_h (\bar{Y}_h - \bar{Y})^2 \\ V_{ran} - V_{opt} &= \frac{1-f}{n} \sum^L_{h=1} W_h (\bar{Y}_h - \bar{Y})^2 + \frac{1}{n} \sum^L_{h=1} W_h (S_h - \bar{S})^2 \end{aligned} $$

I can prove any of this if you would like, but a complete discussion can be found in Elementary Suvey Sampling 7th Ed, R. Scheaffer et al.

Note that:

$$ \begin{aligned} V_{SRS} &= \frac{S^2}{n} (\frac{N-n}{N}) \\ V_{prop} &= \frac{N-n}{Nn} \sum^L_{h=1} W_h S^2_h \\ V_{opt} &= \frac{1}{n} (\sum^L_{h=1} W_h S_h)^2 - \frac{1}{N} \sum^L_{h=1} W_h S^2_h \end{aligned} $$

$\endgroup$
  • $\begingroup$ Thanks @half-pass. I would like to note though, that as soon as you involve different costs for the $L$ strata such that $c_1 \neq c_2 \neq ... \neq c_L$, it is possible that it is not worth it in terms of optimal cost to conduct stratified sampling, so that could get a little bit closer to an answer. $\endgroup$ – Chris C Nov 4 '15 at 15:59
  • 1
    $\begingroup$ Definitely a good point. $\endgroup$ – half-pass Nov 5 '15 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.