# Can Targeted Maximum Likelihood Estimation find the Average Treatment Effect on the Treated (ATT) instead of the Average Treatment Effect (ATE)?

Targeted Maximum Likelihood Estimation (TMLE) has a number of advantages for counterfactual modeling, however in my field we often want to estimate the ATT, rather than the ATE, for health care programs or interventions.

Part of the procedure for finding a TMLE estimate of the ATE is calculating a "clever covariate" for each record in the data:

$$H(A,X)=\frac{I(A=1)}{\hat p(x)}-\frac{I(A=0)}{1-\hat p(x)}$$

The structure of the above formula suggests it's possible to estimate the ATT by simply substituting the inverse weights for ATT:

$$H(A,X)=\frac{I(A=1)\times1}{1}-\frac{I(A=0)\times \hat p(x)}{1-\hat p(x)}$$

Is this a correct assumption or is there another method for estimating the ATT?

Yes TMLE can be used to estimate the ATT and ATC as well. However, different clever covariates are required and, in fact, an iterative procedure is needed that targets both the conditional mean and the propensity score estimators. Note that the R package "tmle" computes the ATE, ATT, and ATC estimates automatically, so that package would be useful.

This book is a good resource: van der Laan MJ, Rose S. Targeted learning: causal inference for observational and experimental data. Springer; New York: 2011. Specifically, Chapter 8 for the ATT TMLE. Note that the estimator below is slightly different (but easier to implement) than the one described in this reference.

For ATT, suppose you observe n i.i.d. observations of the data-structure (W,A,Y) with $$W$$ baseline covariates, binary treatment $$A$$, and an outcome $$Y$$ taking values in $$[0,1]$$. Let $$g_n(W)$$ denote an (initial) estimate of $$P(A=1|W)$$, $$Q_n(a,W)$$ denote an estimate of $$E[Y|A=a,W]$$, and $$P_n(A=1)$$ be an estimate of the marginal probability $$P(A=1)$$. The clever covariate for the conditional mean is actually given by $$H_{n,Y,g_n,P_n}(A,W) = \frac{1(A=1)}{P_n(A=1)} - \frac{1(A=0)g_n(W)}{P_n(A=1)(1-g_n(W))},$$ which is what you wrote but with inverse weighting by the marginal probability of $$A=1$$. It is natural to choose $$P_n(A=1) = \frac{1}{n} \sum_{i=1}^n 1(A_i=1).$$ Clever covariates are also required for the propensity score and are given by $$H_{n,A,Q_n, g_n, P_n}(W) = \frac{Q_n(A=1,W) - Q_n(A=0,W) - \Psi_n(g_n,Q_n,P_n)}{P_n(A=1)}$$ where $$\Psi_n(g_n,Q_n,P_n) = \frac{1}{n} \sum_{i=1}^n \frac{g_n(W_i)}{P_n(A=1)}[Q_n(A=1,W_i) - Q_n(A=0,W_i)]$$ is the plugin estimate for the ATT.

We will now iteratively update/target $$g_n$$ and $$Q_n$$. Start counter $$k=1$$. Let $$g_n^{(k)}$$ and $$Q_n^{(k)}$$ denote the estimators obtained at step $$k$$ with $$k=0$$ corresponding with the initial estimators $$g_n$$ and $$Q_n$$. We now define $$g_n^{(k)}$$ to be the updated estimator obtained by performing the logistic regression of $$A$$ on the clever covariate $$H_{n,A,Q_n^{(k-1)}, g_n^{(k-1)}, P_n}(W)$$ using as offset $$g_n^{(k-1)}(W)$$. Next, we define the updated estimator $$Q_n^{(k)}$$ to be the estimator obtained by performing the logistic regression of $$Y$$ on the clever covariate $$H_{n,Y,g_n^{(k)},P_n}(A,W)$$ using as offset $$Q_n^{(k-1)}(A,W)$$. Keep performing this iterative procedure until the following key score equations are solved $$\frac{1}{n}\sum_{i=1}^n H_{n,Y,g_n^{(k)},P_n}(A_i,W_i)(Y_i - Q_n^{(k)}(A_i,W_i)) \approx 0$$ and $$\frac{1}{n}\sum_{i=1}^n H_{n,A,Q_n^{(k)}, g_n^{(k)}, P_n}(W_i)(A_i - g_n^{(k)}(W_i)) \approx 0$$ (e.g. solve the above score equations up until a level 1/n usually suffices). Once you obtain a value $$k = \kappa$$ that achieves this (usually after only a few iterations), the TMLE ATT estimator is given by $$\Psi_n^* := \frac{1}{n} \sum_{i=1}^n \frac{g_n^*(W_i)}{P_n(A=1)}[Q_n^*(A=1,W_i) - Q_n^*(A=0,W_i)],$$ where $$Q_n^* = Q_n^{(\kappa)}$$ and $$g_n^* = g_n^{(\kappa)}.$$

The standard error of the estimator can be estimated by the empirical standard deviation of $$D_n(Y,A,W) := H_{n,Y,g_n^{*},P_n}(A,W)(Y - Q_n^{*}(A,W)) + H_{n,A,Q_n^{*}, g_n^{*}, P_n}(W)(A - g_n^{*}(W)) + \frac{g_n^*(W)}{P_n(A=1)} [Q_n^*(A=1,W) - Q_n^*(A=0,W)- \Psi_n^*]$$ and a 95% confidence interval is given by $$\Psi_n^* \pm 1.96\frac{sd_n(D_n)}{\sqrt{n}}$$.

Curiously, you can check that $$\frac{1}{n} \sum_{i=1}^n D_n(Y_i,A_i,W_i) \approx 0$$ based on the score equations solved by the TMLE. It turns out that $$D_n$$ is actually an estimate of the efficient influence function of the ATT parameter, and solving this key score equation gives the TMLE its asymptotically normal distribution and efficiency.

• Fantastic, thank you Lars, this is very helpful! After posting my question I did see the tmle package outputs TMLE estimates for ATT - however it's always helpful to see what's going on under the hood, especially if we need to code TMLE into other languages like Python. I have a copy of Targeted Learning but was unable to find a derivation of TMLE for ATT. Jul 14 at 16:12
• BTW I've posted several other questions in Cross Validated on the theory behind TMLE: stats.stackexchange.com/questions/442569/… and stats.stackexchange.com/questions/467969/…. My understanding of how, exactly, influence functions are used to arrive at the "true" population estimate of ATE or ATT using TMLE is still a bit murky - any contributions you could make to answering my questions would be much appreciated! :-) Jul 14 at 16:16
• I am glad it was helpful! I will give my best shot at answering the other questions as well. The ATT derivation of TMLE is somewhat hidden in Chapter 8 of the book (As an indirect/direct effects parameter). The TMLE estimator presented in the book is slightly different than the above. I believe the one I presented agrees with the implementation given in the tmle package and is slightly easier to implement.
– user327671
Jul 14 at 22:42