Why are superlearners used in TMLE?

Question

When estimating causal properties using TMLE, what is the benefit of using superlearners? It seems like there's an assumption that the model that minimizes prediction error is also the best one for causal inference and I don't understand why that is.

For example in this paper, the authors show that the lasso model which is optimal for prediction will contain a lot of noise variables. Therefore minimizing prediction error will increase model misspecification. Isn't it possible that the same thing happens with superlearners, where the model is incorrect but makes good predictions?

Answer 1

Asymptotic inference (i.e., the variance of the estimator) for TMLE using influence functions requires the nuisance models--the models for the expected potential outcomes $E[Y|A,X]$ and propensity scores $E[A|X]$--to converge to the truth (i.e., for the predicted values to converge to the true values) at a certain rate.

Different models converge to the truth at desired rates under a variety of assumptions; for example, if the true propensity score model is a logistic model, maximum likelihood estimated (MLE) logistic regression converges quickly to the truth, but if the true propensity score model is a specific more complicated function, a gradient boosting machine (GBM) may converge at some rate, and the logistic regression won't converge at all. In practice, we don't know what the true model is, so we don't know if we are in a scenario where MLE logistic regression will converge at the required rate or if GBM will.

SuperLearner takes on the fastest convergence rate of its candidate models, which means that to have the best chance of approaching the required converge rate, SuperLearner asymptotically does as well or better than any of its candidate models when the true model is unknown. Using the example above, if the true propensity score model is logistic and MLE logistic regression and GBM are used as candidate libraries for SuperLearner, then the resulting SuperLearner predictions converge to the truth at the same rate as the MLE logistic regression alone would. But if the GBM converges to the truth at a certain rate and the MLE logistic model is wrong, then that same SuperLearner as above will converge at the same rate as the GBM, even with the logistic regression included in the library. So, there is a kind of "multiple robustness" in the SuperLearner in that if any of the candidate models converge to the truth at the required rate, the SuperLearner containing those models does, too.

We know that a machine learning method called "highly adaptive lasso" (HAL) alone converges to the truth at the required rate; so technically, it is the only learner required for asymptotic inference with TMLE to be valid. But if the true propensity score or outcome model is a simple model captured well by MLE logistic regression, we would want predictions from such a model to help steer the predictions to the truth at an even faster rate than HAL would do alone in order to improve precision and arrive at approximately valid inference with a smaller sample size. Including both MLE logistic regression and HAL (and other models) would improve the performance of the resulting predictions while guaranteeing the asymptotic consistency properties imparted by HAL.

Also note that the models are cross-validated and, ideally, cross-fit, which reduces the problems of overfitting that you mention. I believe the convergence rates for a given model refer to their cross-validated rates.

I know others may be able to provide more technical explanations for this, but this is my understanding at an intuitive level of why SuperLearner is so important to TMLE (and AIPW and all other doubly-robust methods).

Answer 2

Minimizing the prediction error of nuisance parameters is (generally) the correct goal when developing efficient estimators using e.g. estimating equations or one-step estimators. To suggest this, let's rigorously focus on a case study: estimating $\mu:=E\{AY/\pi(W)\}$ in a nonparametric model, for $A$ binary and $\pi(w) = P(A=1\mid W=w)$ the propensity score.

Define $Q(a,w) = E(Y \mid A=a, W=w)$ as the outcome regression. The nonparametric efficient influence curve is $$D^*(W,A,Y; P) = \frac{A}{\pi(W)}Y - \frac{A-\pi(W)}{\pi(W)} Q(1,W) - \mu.$$ With i.i.d. sampling, an efficient estimator is $$\hat\mu_n = \frac{1}{n}\sum_{i=1}^n \left\{ \frac{A}{\hat\pi(W)}Y - \frac{A-\hat\pi(W)}{\hat\pi(W)} \hat{Q}(1,W) \right\},$$ whenever cross-fitting is used and the remainder is negligible, i.e. $$E\left[ \frac{\{\hat\pi_n(W) - \pi_0(W)\} \{\hat{Q}_n(1,W) - Q_0(1,W)\}}{\hat\pi_n(W)} \right] = o_p(n^{-1/2}).$$ Since the left hand side is upper bounded by $$\|\hat\pi_n(W) - \pi_0(W)\| \|\hat{Q}_n(1,W) - Q_0(1,W)\|,$$ whenever sample positivity holds, we can see that we should fit the nuisance parameters by ensuring the estimates are close to the truth. Further, recall that minimizing closeness to a true expectation is equivalent to minimizing a prediction error. We can for example accomplish this by minimizing the cross validated error.

Although the TMLE is a different estimator, its influence function is the same and the same considerations apply. We can see this through the following motivation.

The superlearner is nicely described by Noah. It's purpose is to find nuisance parameters which ensure the above norms are as small as possible. As Noah explains, it does this by looking through several estimators and finding which to prefer. From above, we can see that the product of the rate of convergence of both estimators must be $o_p(n^{-1/2})$; since the superlearner inherits the best rate of convergence of its constituent estimators, it can be helpful here.