Can Targeted Maximum Likelihood Estimation find the Average Treatment Effect on the Treated (ATT) instead of the Average Treatment Effect (ATE)? Targeted Maximum Likelihood Estimation (TMLE) has a number of advantages for counterfactual modeling, however in my field we often want to estimate the ATT, rather than the ATE, for health care programs or interventions.
Part of the procedure for finding a TMLE estimate of the ATE is calculating a "clever covariate" for each record in the data:
$$H(A,X)=\frac{I(A=1)}{\hat p(x)}-\frac{I(A=0)}{1-\hat p(x)}$$
The structure of the above formula suggests it's possible to estimate the ATT by simply substituting the inverse weights for ATT:
$$H(A,X)=\frac{I(A=1)\times1}{1}-\frac{I(A=0)\times \hat p(x)}{1-\hat p(x)}$$
Is this a  correct assumption or is there another method for estimating the ATT?
 A: Yes TMLE can be used to estimate the ATT and ATC as well. However, different clever covariates are required and, in fact, an iterative procedure is needed that targets both the conditional mean and the propensity score estimators. Note that the R package "tmle" computes the ATE, ATT, and ATC estimates automatically, so that package would be useful.
This book is a good resource: van der Laan MJ, Rose S. Targeted learning: causal inference for observational and experimental data. Springer; New York: 2011. Specifically, Chapter 8 for the ATT TMLE. Note that the estimator below is slightly different (but easier to implement) than the one described in this reference.
For ATT, suppose you observe n i.i.d. observations of the data-structure (W,A,Y) with $W$ baseline covariates, binary treatment $A$, and an outcome $Y$ taking values in $[0,1]$. Let $g_n(W)$ denote an (initial) estimate of $P(A=1|W)$, $Q_n(a,W)$ denote an estimate of $E[Y|A=a,W]$, and $P_n(A=1)$ be an estimate of the marginal probability $P(A=1)$.
The clever covariate for the conditional mean is actually given by
$$H_{n,Y,g_n,P_n}(A,W) = \frac{1(A=1)}{P_n(A=1)} - \frac{1(A=0)g_n(W)}{P_n(A=1)(1-g_n(W))}, $$
which is what you wrote but with inverse weighting by the marginal probability of $A=1$. It is natural to choose
$$P_n(A=1) = \frac{1}{n} \sum_{i=1}^n 1(A_i=1).$$
Clever covariates are also required for the propensity score and are given by
$$H_{n,A,Q_n, g_n, P_n}(W) = \frac{Q_n(A=1,W)  - Q_n(A=0,W) - \Psi_n(g_n,Q_n,P_n)}{P_n(A=1)}$$
where
$$\Psi_n(g_n,Q_n,P_n) = \frac{1}{n} \sum_{i=1}^n \frac{g_n(W_i)}{P_n(A=1)}[Q_n(A=1,W_i) - Q_n(A=0,W_i)]$$
is the plugin estimate for the ATT.
We will now iteratively update/target $g_n$ and $Q_n$. Start counter $k=1$.
Let $g_n^{(k)}$ and $Q_n^{(k)}$ denote the estimators obtained at step $k$ with $k=0$ corresponding with the initial estimators $g_n$ and $Q_n$. We now define $g_n^{(k)}$ to be the updated estimator obtained by performing the logistic regression of $A$ on the clever covariate $H_{n,A,Q_n^{(k-1)}, g_n^{(k-1)}, P_n}(W)$ using as offset $g_n^{(k-1)}(W)$. Next, we define the updated estimator $Q_n^{(k)}$ to be the estimator obtained by performing the logistic regression of $Y$ on the clever covariate $H_{n,Y,g_n^{(k)},P_n}(A,W)$ using as offset $Q_n^{(k-1)}(A,W)$. Keep performing this iterative procedure until the following key score equations are solved
$$\frac{1}{n}\sum_{i=1}^n H_{n,Y,g_n^{(k)},P_n}(A_i,W_i)(Y_i - Q_n^{(k)}(A_i,W_i)) \approx 0$$
and
$$\frac{1}{n}\sum_{i=1}^n H_{n,A,Q_n^{(k)}, g_n^{(k)}, P_n}(W_i)(A_i - g_n^{(k)}(W_i)) \approx 0$$
(e.g. solve the above score equations up until a level 1/n usually suffices). Once you obtain a value $k = \kappa$ that achieves this (usually after only a few iterations), the TMLE ATT estimator is given by
$$\Psi_n^* := \frac{1}{n} \sum_{i=1}^n \frac{g_n^*(W_i)}{P_n(A=1)}[Q_n^*(A=1,W_i) - Q_n^*(A=0,W_i)],$$
where $Q_n^* = Q_n^{(\kappa)}$ and $g_n^* = g_n^{(\kappa)}.$
The standard error of the estimator can be estimated by the empirical standard deviation of
$$D_n(Y,A,W) :=  H_{n,Y,g_n^{*},P_n}(A,W)(Y - Q_n^{*}(A,W)) + H_{n,A,Q_n^{*}, g_n^{*}, P_n}(W)(A - g_n^{*}(W)) +  \frac{g_n^*(W)}{P_n(A=1)} [Q_n^*(A=1,W) - Q_n^*(A=0,W)- \Psi_n^*]$$
and a 95% confidence interval is given by $\Psi_n^* \pm 1.96\frac{sd_n(D_n)}{\sqrt{n}}$.
Curiously, you can check that $\frac{1}{n} \sum_{i=1}^n D_n(Y_i,A_i,W_i) \approx 0 $ based on the score equations solved by the TMLE. It turns out that $D_n$ is actually an estimate of the efficient influence function of the ATT parameter, and solving this key score equation gives the TMLE its asymptotically normal distribution and efficiency.
