2
$\begingroup$

What effect is estimated from the execution of IPW?

When running IPW in R, I know that you specify the formula for the weight itself, i.e.:

(treatment / propensity) + ((1 - treatment) / (1 - propensity))

To my knowledge, this weight helps generate the ATE.

My question is this, is the ATE what you ideally should be estimating when using IPW? This might seem like a simple question since most of the implementations of IPW I have seen estimate the ATE. However, I am slightly confused by the wording of a 2008 Paper which uses matching to estimate the causal effect of UN peacekeeping operations on peace in which the authors claim:

"By necessity we estimate the average treatment effect on the treated (ATT). That is, we answer the question: how well did the United Nations do in the range of cases where it actually intervened and, if the United Nations were to intervene in a similar case, how well would we expect it to perform? We cannot tell how well the United Nations would have done had it intervened in a type of situation where it has never intervened (typically called the average treatment effect on the control or ATC). We do not have treated cases to match to all of our control cases and so we cannot answer that question. Like drugs and surgical procedures, United Nations interventions are designed for certain cases and not for others."

I know that the authors here are not using IPW, however, I thought the point of IPW was to balance the data set to account for specified confounders that create an imbalance with respect towards treatment propensity in the first place.

According to a response on this Stack Overflow post, the ATE and ATT are the same when "the baseline of the treatment group equals the baseline of the control group [and] the treatment effect on the treated group equals the treatment effect on the control group". We cannot necessarily know this with observational data, but I thought, in balancing the data set, we can get closer to meeting these assumptions so that we can estimate ATE.

I very well may be mistaken here as I am still trying to wrap my head around when different treatment effects are appropriate. My fundamental area of confusion is the degree to which IPW allows one (if it does at all) to estimate the ATE with observational data due to its ability to balance the data set.

$\endgroup$
3
  • $\begingroup$ Hey, Brian! It seems like much of your confusion stems around the usefulness of IPW. I would attempt to delineate this for you; however, two much brighter and more eloquent people have done this here: cdn1.sph.harvard.edu/wp-content/uploads/sites/1268/2022/12/…. See section 2.4 on page 20. This reference is actually a great introduction to graduate-level causal inference! $\endgroup$
    – Ron Snow
    Jan 2, 2023 at 17:25
  • $\begingroup$ I don't want to leave you hanging; once you read that section, let me know more where any confusion still persists. In general, IPW can be used to estimate several statistics of interest. It just so happens that ATE is the gold standard in causal inference. That is, if we can obtain a good estimate for the ATE, then we certainly should! However, several assumptions must be made in terms of the problem and the causality framework that we are considering to make inference. Whether those assumptions are met should be carefully analyzed and conveyed in papers or to collaborators. $\endgroup$
    – Ron Snow
    Jan 2, 2023 at 17:28
  • $\begingroup$ Also, balancing the data set (whether or not you use IPW), does not necessarily change how close the baselines and effects are between groups. The post you linked is correct, but that situation is extremely rare in the wild and hard to force in any framework without violating some causal inference sins. Instead, what IPW is useful for is (hopefully) correctly weighting observations in groups to be more similarly distributed to the other group. This way, the groups are more similar; but they will rarely be identical, so rarely will ATE and ATT be identical. Yet, ATT may be a good approximation. $\endgroup$
    – Ron Snow
    Jan 2, 2023 at 17:31

1 Answer 1

2
$\begingroup$

It seems like you have several questions, so please correct me if I'm wrong.

What effect is estimated when using IPW?

It depends on how you construct the weights. There is a formula for each of the main estimands, the ATE (average treatment effect in the population), ATT (average treatment effect on the treated), ATC (average treatment effect on the control), or ATO (average treatment effect in the overlap). There are yet other formulas for computing weights from propensity scores that target other populations and have different proprties. See Li for a general theory.

The usual IPW weights, computed using the formula you wrote, targets the ATE. What these weights do when correctly specified is to make the distribution of covariates in the treated group and the distribution of covariates in the control group resemble the distribution of covariates in the full sample (and thereby, resemble each other).

Of course, the ATE is only identified when all causes of the outcome are balanced by the weights. In observational studies, we don't know whether we have balanced all the causes of the outcome because not all of the have been measured.

Is the ATE what you should be estimating with IPW?

That depends on if you want the ATE. You need to choose the estimand that is most substantively relevant (see Greifer & Stuart (2021) for advice on how to make this choice) and then use the formula for that estimand to compute the weights.

The ATT and the ATE are equal when there is no effect modification by the confounders OR when the distribution of effect modifiers is the same between the treated and control group prior to weighting. One should never count on these assumptions since they require a lot of evidence to validate. You should always choose the estimand and compute the weights for it assuming the estimands target different effects.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.