I read in this paper An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies, that

"If the outcome is dichotomous (self-report of the presence or absence of depression), the effect of treatment can be estimated as the difference between the proportion of subjects experiencing the event in each of the two groups (treated vs. untreated) in the matched sample. With binary outcomes, the effect of treatment can also be described using the relative risk..."

It seems to me that the difference of the proportions of subjects experiencing the event under treatment and non-treatment refers to the estimator for the Average Treatment Effect on the Treated (ATT), instead of the Average Treatment Effect (ATE).

Is calculating the ATE impossible under a binary outcome?

  • $\begingroup$ From the quoted excerpt I take away that for binary outcomes you can compute the treatment effect estimates from sample proportions. For dose response you clearly wouldn't look at proportions. Where do you take the ATT vs ATE argument from? $\endgroup$ Mar 4, 2018 at 9:55
  • $\begingroup$ You are ignoring outcome heterogeneity (e.g., non-collapsibility of odds ratios) and assuming that matching is a good idea. There are many negatives to matching as discussed in fharrell.com/doc/bbr.pdf which also contains a statistical analysis plan that uses regression adjustment for propensity score in addition to the most important pre-specified predictors. $\endgroup$ Mar 4, 2018 at 14:24
  • $\begingroup$ @FrankHarrell Can you elaborate on what you mean by outcome heterogeneity? Are you saying that looking at the relative risk is bad in principle? $\endgroup$
    – user321627
    Mar 9, 2018 at 5:20
  • $\begingroup$ No, I meant that adjusting for heterogeneity in treatment selection does not adjust for outcome heterogeneity. For a completely proper analysis you need to have the major predictive covariates in the model along with the propensity score. I have details about this in fharrell.com/doc/bbr.pdf . Ignoring this gives treatment effects too close to the null by making the model not fit. $\endgroup$ Mar 9, 2018 at 13:28

1 Answer 1


Calculating the ATE under a binary outcome is possible. That being said, yes, you are correct to suspect that through Propensity Score Matching (PSM) we are probably looking an ATT estimate or an even more constrained treatment effect because we might not have good overlap (common support) between the two treatment groups.

The passage you quote, correctly implies that a reasonable estimate of ATE is obtainable; it explicitly mentions that this calculation takes place "in the matched sample". To that respect, the opening paragraph of the section the passage comes from also states that: "(Thus), in a set of subjects all of whom have the same propensity score, the distribution of observed baseline covariates will be the same between the treated and untreated subjects.". i.e. ATE estimates are theoretically coherent.

I think that the best way to consider ATT/ATC/ATE estimates is by examining them within a counterfactual framework that is relevant to our research question. That way we can assess in a straight-forward manner what each difference refers at. We state the assumptions made (e.g. $E[Y^0|A = 1, X] = E[Y^0|A= 0, X] \approx$ if people in treatment group $A=1$ would do "as bad" as the control group $A=0$ if they were not treated, given information on covariates $X$) and then we calculate the difference we care for. Generally assessing "differences in the groups" can be potentially misinterpreted.

Morgan & Winship's "Counterfactuals and Causal Inference", Chapt. 5 "Matching Estimators of Causal Effects" has a very smooth exposition of the subject.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.