35
$\begingroup$

In RCTs, randomisation balances unmeasured confounders and, I'm told, ATE and ATT would be the same. In observational studies, this is not possible and Propensity Scores are used in various ways to estimate ATT and/or ATE. The analyses that I've performed and examples that I've seen (eg this helpful text) shows different ATT and ATE (albeit slightly).

Please can anyone help me understand why they are different and, more importantly, what the differences mean (eg. if ATE>ATT or ATT>ATE), if anything?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
61
$\begingroup$

The Average Treatment Effect (ATE) and the Average Treatment Effect on Treated (ATT) are commonly defined across the different groups of individuals. In addition, ATE and ATT are often different because they might measure outcomes ($Y$) that are not affected from the treatment $D$ in the same manner.

First, some additional notation:

  • $Y^0$: population-level random variable for outcome $Y$ in control state.
  • $Y^1$: population-level random variable for outcome $Y$ in treatment state.
  • $\delta$: individual-level causal effect of the treatment.
  • $\pi$: proportion of population that takes treatment.

Given the above, the ATT is defined as: $\mathrm{E}[\delta|D=1]$ ie. what is the expected causal effect of the treatment for individuals in the treatment group. This can be decomposed more meaningfully as: \begin{align} \mathrm{E}[\delta|D=1] = & \mathrm{E}[Y^1 - Y^0|D=1] \\ & \mathrm{E}[Y^1|D=1] - \mathrm{E}[Y^0|D=1] \end{align}

(Notice that $\mathrm{E}[Y^0|D=1]$ is unobserved so it refers to a counterfactual variable which is not realised in our observed sample.) Similarly the ATE is defined as: $\mathrm{E}[\delta]$, ie. what is the expected causal effect of the treatment across all individuals in the population. Again we can decompose this more meaningfully as: \begin{align} \mathrm{E}[\delta] =& \{ \pi \mathrm{E}[Y^1|D=1] + (1-\pi) \mathrm{E}[Y^1|D=0] \} \\ -& \{ \pi \mathrm{E}[Y^0|D=1] + (1-\pi) \mathrm{E}[Y^0|D=0] \} \end{align}

As you see the ATT and the more general ATE are referring by definition to different portions of the population of interest. More importantly, in the ideal scenario of a randomised control trial (RCT) ATE equals ATT because we assume that:

  • $\mathrm{E}[Y^0|D=1] = \mathrm{E}[Y^0|D=0]$ and
  • $\mathrm{E}[Y^1|D=1] = \mathrm{E}[Y^1|D=0]$,

ie. we have believe respectively that:

  • the baseline of the treatment group equals the baseline of the control group (layman terms: people in the treatment group would do as bad as the control group if they were not treated) and
  • the treatment effect on the treated group equals the treatment effect on the control group (layman terms: people in the control group would do as good as the treatment group if they were treated).

These are very strong assumptions which are commonly violated in observational studies and therefore the ATT and the ATE are not expected to be equal. (Notice that if only the baselines are equal, you can still get an ATT through simple differences: $\mathrm{E}[Y^1|D=1] - \mathrm{E}[Y^0|D=0]$.)

Especially in the cases where the individuals self-select to enter the treatment group or not (eg. an e-shop providing cash bonus where a customer can redeem a bonus coupon for $X$ amount given she shops items worth at least $Y$ amount) the baselines as well as the treatment effects can be different (eg. repeat buyers are more likely to redeem such a bonus, low-value customers might find the threshold $Y$ unrealistically high or high-value customers might be indifferent to the bonus amount $X$ - this also relates to SUTVA). In scenarios like this even talking about ATE is probably ill-defined (eg. it is unrealistic to expect that all the customers of an e-shop will ever shop items worth $Y$).

ATT being unequal to ATE is not unexpected. If ATT is smaller or greater than ATE is application specific. The inequality of the two suggests that the treatment assignment mechanism was potentially not random. In general, in an observational study because the above-mentioned assumptions do not generally hold, we either partition our sample accordingly or we control for difference through "regression-like" techniques.

For a more detailed but easy to follow exposition of the matter I recommend looking into Morgan & Winship's Counterfactuals and Causal Inference.

Improve this answer
$\endgroup$
12
  • 2
    $\begingroup$ Thank you very much for this incredibly detailed and helpful answer. I'm not a statistician and struggle at times with formulae, but this is very clear. Will Morga & Winship's book be digestible by a layman, or can you suggest a "dummy's guide" to causal inference? Thanks again $\endgroup$
    – bobmcpop
    Commented Oct 25, 2017 at 2:30
  • 1
    $\begingroup$ I am glad I could help. I have educated myself mostly from papers so I have a limited view of what causal inference books are out there. That said, I have found M&W's book to be clear and easy to comprehend; I think an inclined layman will have little problems to follow through. The book is part of the "Analytical Methods for Social Research" series from Cambridge Univ. Press so it uses mostly Sociology-based examples. @DimitriyV.Masterov might have a more educated suggestion. $\endgroup$
    – usεr11852
    Commented Oct 25, 2017 at 23:42
  • $\begingroup$ Thanks, I'll get myself a copy. "The inequality of the two suggests that the treatment assignment mechanism was potentially not random." I assume in a hypothetical situation where literally every baseline confounder was measured in an observational study, and there was a perfect match for each PS, we would get very close to those assumptions. Therefore would the extent to which ATT/ATE are discordant provide any meaningful information about how poorly the PS balanced for unmeasured confounders? $\endgroup$
    – bobmcpop
    Commented Nov 7, 2017 at 14:24
  • 1
    $\begingroup$ In a hypothetical situation, yes. I think it would be meaningful in the context of a simulation study. That said, actual using it to quantify "poorness/goodness" of balance achieved by PS is probably a methodical exercise on its own right. (Happy reading!) $\endgroup$
    – usεr11852
    Commented Nov 7, 2017 at 20:56
  • 1
    $\begingroup$ @bobmcpop I'm a statistician, and I never have understood why someone would want to measure the ATT instead of the ATE. It's important to have a control group to account for regression to the mean effects & other factors - you're missing this when you use the ATT. $\endgroup$
    – RobertF
    Commented Oct 18, 2018 at 20:42
13
$\begingroup$

ATE is the average treatment effect, and ATT is the average treatment effect on the treated.

The ATT is the effect of the treatment actually applied. Medical studies typically use the ATT as the designated quantity of interest because they often only care about the causal effect of drugs for patients that receive or would receive the drugs.

For another example, ATT tells us how much the typical soldier gained or lost as a consequence of military service, while ATE tells us how much the typical applicant to the military gained or lost.

Improve this answer
$\endgroup$
4
  • $\begingroup$ You're confusing the ATT with the ITT, intent-to-treat effect. $\endgroup$
    – Noah
    Commented Mar 21, 2020 at 3:43
  • 2
    $\begingroup$ I don't confuse. I refer to this paper: Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political analysis, 15(3), 199-236. $\endgroup$
    – Yao Zhao
    Commented Mar 21, 2020 at 15:51
  • 2
    $\begingroup$ You're right, I misunderstood. $\endgroup$
    – Noah
    Commented Mar 21, 2020 at 18:20
  • 3
    $\begingroup$ This is one of the clearest explanations I have seen of ATE vs ATT $\endgroup$
    – asdfkjasdf33
    Commented May 14, 2020 at 14:29
0
$\begingroup$

There seems to be something obvious not discussed above. If the treatment effect is constant over individuals $Y_i^1-Y_i^0=c$ for every $i$ then ATT and ATE as defined above should be equal.

This may be seen as there being no "effect modifiers". My intuition is that balancing effect modifiers between groups would also be sufficient for equality of ATT and ATE. The distribution of treatment effects would then be independent of treatment. Maybe someone already has this formalized?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.