For the treatment assignment vector in causal inference, what is the difference between assuming a superpopulation vs. a finite-sample population?

In causal inference, there is a very subtle difference between viewing the potential outcomes $Y(1), Y(0)$ as coming from a super-population or a finite population. Specifically, in a super-population, we assume that $Y(1), Y(0)$ are stochastic and hence induce a distribution onto the treatment vector, $Z$. In the finite-sample population, the potential outcomes are fixed. In this case, how does the treatment assignment vector vary from that of the super-population? Is the general way to think about it in that one has a distribution while the other doesn't?

• I added survey-sampling to attract interest of people who understand the finite sampling paradigms and inference. Commented Oct 30, 2017 at 2:49
• What is finite-sample population and is it different from population? Does it cover both finite and infinite populations? Commented Oct 30, 2017 at 6:23
• Hi, user, just checking back here --- do you think the answer below helped/is clear? Commented Mar 14, 2018 at 5:34
• Hi Carlos, definitely! I chose it as the best answer, thanks! Commented Mar 14, 2018 at 5:39

For the super population, you don't need to see the potential outcomes as stochastic, neither as infinite. The difference is basically this: the "finite-sample" treatment effect is the causal effect for the specific sample you have, the only uncertainty in your estimate will come from the randomization of the treatment assignment. The "super population" treatment effect is the causal effect of a target population your sample might have come from --- so now you have two sources of uncertainty in your estimate: the randomization of the treatment assignment, and the fact that what you have is a sample from your target population, not the whole population.

To make things easier, imagine a "super population" of size $N_{sp} = 100$. For simplicity, imagine the potential outcomes of this population are $Y_i(1) = i$ for individuals $i$ from 1 to 100, and $Y_i(0) = 0$ for all $i$. Hence the average causal effect is $\tau_{sp} = N_{sp}^{-1}\sum_{i =1}^{N_{sp}}(Y_i(1) - Y_i(0)) = 50.5$ on the "super population".

Now suppose you have a sample of $N_{fs} = 10$ people from this population. Imagine you have for some reason sampled the values: 6 13 21 48 53 64 72 79 94 100. Hence, the "finte sample" average causal effect is $\tau_{fs} = N_{fp}^{-1}\sum_{i=1}^{N_{sp}}R_i(Y_i(1) - Y_i(0)) = 55$, where $R_i$ is an indicator variable which equals one if the individual was sampled.

You can connect the "finite sample average treatment effect" with the "super population average treatment effect" if you know how the sampling process took place. Assuming it was a simple random sample, for instance, then:

\begin{aligned} E[\tau_{fs}] &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}E[R_i](Y_i(1) - Y_i(0))\\ &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}\frac{N_{fp}}{N_{sp}}(Y_i(1) - Y_i(0))\\ &= \frac{1}{N_{sp}}\sum_{i =1}^{N_{sp}}(Y_i(1) - Y_i(0))= \tau_{sp} \end{aligned}

That is, under this assumption about the sampling process, the average you would get for $\tau_{fs}$, if you repeatedly took samples from your super population, would equal $\tau_{sp}$. Notice that here we are only talking about "true" causal quantities we never see. Now let's connect them to some estimator, for example, the traditional difference in means estimator.

First, to estimate $\tau_{fp}$ you will need to randomize the treatment assignment $T_i$ and you only see $Y_i(1)$ for those who were treated and $Y_i(0)$ for those who were not treated (control). Let $N_t$ be the number of individuals on the treated group and $N_c$ the number of individuals on the control group, with the probability of treatment $E[T_i] = \frac{N_t}{N_{fp}}$. Then your estimate would be:

$$\hat{\tau}_{dif} = \frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i T_i Y_i - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i (1-T_i) Y_i$$

If you take the expectation over $T_i$, considering $R$ and the potential outcomes fixed, you see that: \begin{aligned} E_{T}[\hat{\tau}_{dif}] &= \frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i E[T_i] Y_i - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i E[(1-T_i)] Y_i\\ &=\frac{1}{N_t}\sum_{i =1}^{N_{sp}} R_i \frac{N_t}{N_{fp}} Y_i(1) - \frac{1}{N_c}\sum_{i =1}^{N_{sp}} R_i \frac{N_c}{N_{fp}} Y_i(0)\\ &= \frac{1}{N_{fp}}\sum_{i=1}^{N_{sp}}R_i(Y_i(1) - Y_i(0))= \tau_{fp} \end{aligned}.

Now, if you take again the expectation over the sampling scheme from the superpopulation, that is, $R_i$, as we have shown, you recover $\tau_{sp}$. So you can see $\hat{\tau}_{dif}$ both as an estimate of $\tau_{fp}$ and $\tau_{sp}$ for this simple case. The variances with respect to them will differ though, it's easy to see that we have a noisier estimate of $\tau_{sp}$, so the variance with respect to that target will be higher. Notice what licensed us to do these things is that we are assuming we know both the randomization schemes of the treatment assignment $T_i$ and of how our sampling process from the target super population $R_i$.

• Could you elaborate here on why you took potential outcomes to be fixed? If potential outcomes were not fixed, say it depended on the covariates, how would the above change? Commented Oct 22, 2018 at 17:45
• @user321627 the potential outcome does depend on the covariates! It is fixed for a given individual (assuming SUTVA), because for any given individual his covariates are fixed. But had the individual covariates been different, his/her potential outcome could have been different. Commented Oct 23, 2018 at 2:05
• Does the above derivations and results only hold in the randomized trial scenario or does it hold in the observational study setting? Specifically how you defined "finite-sample" and "super population". Thanks! Commented Apr 1, 2019 at 5:18