For the treatment assignment vector in causal inference, what is the difference between assuming a superpopulation vs. a finite-sample population?

Question

In causal inference, there is a very subtle difference between viewing the potential outcomes $Y(1), Y(0)$ as coming from a super-population or a finite population. Specifically, in a super-population, we assume that $Y(1), Y(0)$ are stochastic and hence induce a distribution onto the treatment vector, $Z$. In the finite-sample population, the potential outcomes are fixed. In this case, how does the treatment assignment vector vary from that of the super-population? Is the general way to think about it in that one has a distribution while the other doesn't?

Answer 1

For the super population, you don't need to see the potential outcomes as stochastic, neither as infinite. The difference is basically this: the "finite-sample" treatment effect is the causal effect for the specific sample you have, the only uncertainty in your estimate will come from the randomization of the treatment assignment. The "super population" treatment effect is the causal effect of a target population your sample might have come from --- so now you have two sources of uncertainty in your estimate: the randomization of the treatment assignment, and the fact that what you have is a sample from your target population, not the whole population.

To make things easier, imagine a "super population" of size $N_{sp} = 100$. For simplicity, imagine the potential outcomes of this population are $Y_i(1) = i$ for individuals $i$ from 1 to 100, and $Y_i(0) = 0$ for all $i$. Hence the average causal effect is $\tau_{sp} = N_{sp}^{-1}\sum_{i =1}^{N_{sp}}(Y_i(1) - Y_i(0)) = 50.5$ on the "super population".

Now suppose you have a sample of $N_{fs} = 10$ people from this population. Imagine you have for some reason sampled the values: 6 13 21 48 53 64 72 79 94 100. Hence, the "finte sample" average causal effect is $\tau_{fs} = N_{fp}^{-1}\sum_{i=1}^{N_{sp}}R_i(Y_i(1) - Y_i(0)) = 55$, where $R_i$ is an indicator variable which equals one if the individual was sampled.

You can connect the "finite sample average treatment effect" with the "super population average treatment effect" if you know how the sampling process took place. Assuming it was a simple random sample, for instance, then:

$$ \begin{aligned} E[\tau_{fs}] &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}E[R_i](Y_i(1) - Y_i(0))\\ &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}\frac{N_{fp}}{N_{sp}}(Y_i(1) - Y_i(0))\\ &= \frac{1}{N_{sp}}\sum_{i =1}^{N_{sp}}(Y_i(1) - Y_i(0))= \tau_{sp} \end{aligned} $$

That is, under this assumption about the sampling process, the average you would get for $\tau_{fs}$, if you repeatedly took samples from your super population, would equal $\tau_{sp}$. Notice that here we are only talking about "true" causal quantities we never see. Now let's connect them to some estimator, for example, the traditional difference in means estimator.

First, to estimate $\tau_{fp}$ you will need to randomize the treatment assignment $T_i$ and you only see $Y_i(1)$ for those who were treated and $Y_i(0)$ for those who were not treated (control). Let $N_t$ be the number of individuals on the treated group and $N_c$ the number of individuals on the control group, with the probability of treatment $E[T_i] = \frac{N_t}{N_{fp}}$. Then your estimate would be:

$$ \hat{\tau}_{dif} = \frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i T_i Y_i - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i (1-T_i) Y_i $$

If you take the expectation over $T_i$, considering $R$ and the potential outcomes fixed, you see that: $$ \begin{aligned} E_{T}[\hat{\tau}_{dif}] &= \frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i E[T_i] Y_i - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i E[(1-T_i)] Y_i\\ &=\frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i \frac{N_t}{N_{fp}} Y_i(1) - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i \frac{N_c}{N_{fp}} Y_i(0)\\ &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}R_i(Y_i(1) - Y_i(0))= \tau_{fp} \end{aligned} $$.

Now, if you take again the expectation over the sampling scheme from the superpopulation, that is, $R_i$, as we have shown, you recover $\tau_{sp}$. So you can see $\hat{\tau}_{dif}$ both as an estimate of $\tau_{fp}$ and $\tau_{sp}$ for this simple case. The variances with respect to them will differ though, it's easy to see that we have a noisier estimate of $\tau_{sp}$, so the variance with respect to that target will be higher. Notice what licensed us to do these things is that we are assuming we know both the randomization schemes of the treatment assignment $T_i$ and of how our sampling process from the target super population $R_i$.