Sandwich variance estimator or bootstrap-based variance for stabilized inverse probability weighting (IPW)

Question

Multiple published papers describe IPW as akin to having population with multiply copies of the same individuals. Hence, the correlation should be accounted and corrected using sandwich variance estimator or bootstrap-based variance in subsequent analysis when weights are supplied. In all honesty, I do not buy this as it is a pseudo-population not actual population.

With stabilized IPTW weights of binary exposure, total sample size is the same pre-weighting (i.e. there is no artificial inflating of sample size). Does that mean using sandwich variance estimator or bootstrap-based variance is not required? If yes, is it required for binary outcomes?

Answer 1

Every estimator has a variance, and we can estimate that variance from our data. Instead of thinking about pseudo-populations and sample sizes, think about whether a procedure yields a good estimate of an estimator's variance. Statistical theory tells us what the true variance is and often leads us in the direction of good estimators of that variance. For IPW, we have good estimators of the variance for many cases. However, it is important to know that there is no "one" IPW estimate; the estimate depends on how the weights are used, e.g., in a Horvitz-Thompson estimator, in a Hajek estimator, in weighted least squares or maximum likelihood, in a fixed effects or random effects model, etc., as well as on how the weights were computed, e.g., whether they are stabilized or not, which estimand they target, which method was used to estimate them, etc. There is no single general theory that applies to all possible ways of using the weights and all ways of estimating them.

For the most common and simplest case of using IPW in a weighted regression of the outcome on the treatment, we have some good estimators of the variance of the treatment effect. Again, you should not rely on your intuition about sample size and pseudo-populations to try to decide which variance is correct; there are specific formulas and methods that you can use that statistical theory and empirical research have determined are effective.

Bootstrapping tends to work well in practice regardless of how the weights are estimated or used. The robust sandwich estimator that treats the weights as fixed is easy to implement and quick, but it can either be conservative or anti-conservative depending on the specific features of the weights (e.g., which estimand is targeted). The M-estimation-based estimator that accounts for estimation of the weights can be effective but isn't available for all weighting methods (e.g., those that use machine learning) and is only correct asymptotically, so doesn't always perform well in small samples.

You implied considering the usual variance resulting from a weighted maximum likelihood when using stabilized weights. There is little evidence to support using this variance estimator. The fact that stabilizing the weights "preserves" the original sample size has nothing to do with whether this estimator of the variance is valid or not; it has no statistical theory to support it, so there is no reason to use it, even if it can yield similar results to the other variance estimators above.

Answer 2

Viewing it as multiple copies of individuals is helpful to conceptualize what IPW is doing (i.e., constructing a pseudo-population), but maybe less helpful for understanding the variance. To see why, consider the following two cases: (1) the propensity score is known, and (2) the propensity score is unknown and must be estimated.

Case 1: Propensity score is known

When the propensity score is known, we can simply use its value. Here, think of a randomized trial. Because the trial has a known treatment assignment mechanism, we know the true propensity score. So, we can simply use the IPW estimator and plug in the known values for all the units. When estimating the variance, we do not need to account for the 'multiple' copies. The simplest case is a 1:1 trial, so the unstabilized IPW would be 2 for everyone. But we don't need to account for the 'two' copies of everyone.

Case 2: Propensity score is unknown

When the propensity score is unknown, as in an observational study, we might instead try to estimate it. As the propensity score is being estimated, it has its own associated uncertainty. So, our variance estimator needs to incorporate both the uncertainty in our parameter of interest and uncertainty in the propensity score model parameters (i.e., the nuisance parameters).

To emphasize the distinction between the cases, it is that case 2 involves estimation of additional parameters. Hence we need to account for those. Both IPW estimators reweight individuals (regardless of whether the IPW are stabilized or not), so the important distinction is what exactly is being estimated (and has an associated uncertainty).

A source of possible confusion

In the IPW literature, there is a trick for variance estimation. Essentially, you can use the sandwich variance estimator that ignores the uncertainty for the propensity score model (case 2). Let's call this the GEE trick. Weirdly this variance estimator is conservative for the average treatment effect (ATE), which is not what you would normally expect. But it does work. Due to its ease of implementation through software for GEE, it has become a popular approach. However, the overlap of ideas (correlation between observations due to copies, GEE being used to account for correlated observations) I think people tend to confuse the ideas.

What about the bootstrap?

While the bootstrap is more computationally demanding, it is not conservative. So, when statistical power is important the bootstrap might be preferred when in case 2 and sample sizes are limited. A computationally simpler approach is to use the sandwich variance estimator but with the propensity score model and IPW estimator stacked together (traditionally software does not use this approach). For details on this, see the further readings.

Summary

The important part to consider is what is being estimated. When the propensity score is known, the variance estimator does not need to incorporate the uncertainty of the propensity score (because there is none). When the propensity score is being estimated, we need to account for that uncertainty. This can be done by using the GEE trick, bootstrapping, or by stacking the estimating equations and using the sandwich variance estimator.

Further readings

Reifeis, Sarah A., and Michael G. Hudgens. "On variance of the treatment effect in the treated when estimated by inverse probability weighting." American Journal of Epidemiology 191.6 (2022): 1092-1097. LINK

Ross, R. K., Zivich, P. N., Stringer, J. S., & Cole, S. R. (2024). M-estimation for common epidemiological measures: introduction and applied examples. International Journal of Epidemiology, 53(2), dyae030. LINK