Why are superlearners used in TMLE?

When estimating causal properties using TMLE, what is the benefit of using superlearners? It seems like there's an assumption that the model that minimizes prediction error is also the best one for causal inference and I don't understand why that is.

For example in this paper, the authors show that the lasso model which is optimal for prediction will contain a lot of noise variables. Therefore minimizing prediction error will increase model misspecification. Isn't it possible that the same thing happens with superlearners, where the model is incorrect but makes good predictions?

• My intuitive guess is that because the super learner uses the output of several different machine learning models, it is less likely to carry the problems of each one of them into the predictions. It might be an analogous reasoning to why you use the mean to summarize the distribution of your data instead of picking a single observation. Commented Jan 9, 2023 at 3:07
• I'm not sure how the attached paper is related, but I agree with your observation that optimizing for the predictive parameter $E[Y|X,A]$ is not the same as optimizing for the causal parameter $E[Y^1-Y^0|X,A]$. However, truly optimizing for prediction (i.e., not via overfitting) is just the first step of TMLE, which then also uses the "clever covariate" to explain whatever residual information is left and by doing so shifts the focus from a predictive parameter to a causal parameter. Commented Jan 10, 2023 at 20:04
• Is it possible to over-optimize? My thinking is that the relationship between the predictive parameter and the causal parameter isn't linear. It's possible to overshoot and any further optimization will improve the predictive parameter but will move the estimate of the causal parameter further from the truth. Commented Jan 10, 2023 at 20:16

Asymptotic inference (i.e., the variance of the estimator) for TMLE using influence functions requires the nuisance models--the models for the expected potential outcomes $$E[Y|A,X]$$ and propensity scores $$E[A|X]$$--to converge to the truth (i.e., for the predicted values to converge to the true values) at a certain rate.

Different models converge to the truth at desired rates under a variety of assumptions; for example, if the true propensity score model is a logistic model, maximum likelihood estimated (MLE) logistic regression converges quickly to the truth, but if the true propensity score model is a specific more complicated function, a gradient boosting machine (GBM) may converge at some rate, and the logistic regression won't converge at all. In practice, we don't know what the true model is, so we don't know if we are in a scenario where MLE logistic regression will converge at the required rate or if GBM will.

SuperLearner takes on the fastest convergence rate of its candidate models, which means that to have the best chance of approaching the required converge rate, SuperLearner asymptotically does as well or better than any of its candidate models when the true model is unknown. Using the example above, if the true propensity score model is logistic and MLE logistic regression and GBM are used as candidate libraries for SuperLearner, then the resulting SuperLearner predictions converge to the truth at the same rate as the MLE logistic regression alone would. But if the GBM converges to the truth at a certain rate and the MLE logistic model is wrong, then that same SuperLearner as above will converge at the same rate as the GBM, even with the logistic regression included in the library. So, there is a kind of "multiple robustness" in the SuperLearner in that if any of the candidate models converge to the truth at the required rate, the SuperLearner containing those models does, too.

We know that a machine learning method called "highly adaptive lasso" (HAL) alone converges to the truth at the required rate; so technically, it is the only learner required for asymptotic inference with TMLE to be valid. But if the true propensity score or outcome model is a simple model captured well by MLE logistic regression, we would want predictions from such a model to help steer the predictions to the truth at an even faster rate than HAL would do alone in order to improve precision and arrive at approximately valid inference with a smaller sample size. Including both MLE logistic regression and HAL (and other models) would improve the performance of the resulting predictions while guaranteeing the asymptotic consistency properties imparted by HAL.

Also note that the models are cross-validated and, ideally, cross-fit, which reduces the problems of overfitting that you mention. I believe the convergence rates for a given model refer to their cross-validated rates.

I know others may be able to provide more technical explanations for this, but this is my understanding at an intuitive level of why SuperLearner is so important to TMLE (and AIPW and all other doubly-robust methods).

• The claim that the HAL is all you need seems does not seem spiritually accurate to me. I have never understood why their result does not violate the minimax estimation rates known for smooth functions, and either way must be making hidden assumptions to get around the curse of dimensionality.
– guy
Commented Jan 10, 2023 at 23:00
• @guy the theory for HAL does not assume local smoothness - instead it assumes a global finite sectional variation norm.
– Ben
Commented Jan 11, 2023 at 0:02
• @Ben I don't see why that makes the situation fundamentally easier. Generally one expects that for less smooth function spaces you get slower rates, not faster ones.
– guy
Commented Jan 11, 2023 at 0:23
• Hi @guy I suppose the situation is that the function space isn't more smooth or less smooth - its just a different space defined instead by the sectional variation norm
– Ben
Commented Jan 11, 2023 at 19:30

Minimizing the prediction error of nuisance parameters is (generally) the correct goal when developing efficient estimators using e.g. estimating equations or one-step estimators. To suggest this, let's rigorously focus on a case study: estimating $$\mu:=E\{AY/\pi(W)\}$$ in a nonparametric model, for $$A$$ binary and $$\pi(w) = P(A=1\mid W=w)$$ the propensity score.

Define $$Q(a,w) = E(Y \mid A=a, W=w)$$ as the outcome regression. The nonparametric efficient influence curve is $$D^*(W,A,Y; P) = \frac{A}{\pi(W)}Y - \frac{A-\pi(W)}{\pi(W)} Q(1,W) - \mu.$$ With i.i.d. sampling, an efficient estimator is $$\hat\mu_n = \frac{1}{n}\sum_{i=1}^n \left\{ \frac{A}{\hat\pi(W)}Y - \frac{A-\hat\pi(W)}{\hat\pi(W)} \hat{Q}(1,W) \right\},$$ whenever cross-fitting is used and the remainder is negligible, i.e. $$E\left[ \frac{\{\hat\pi_n(W) - \pi_0(W)\} \{\hat{Q}_n(1,W) - Q_0(1,W)\}}{\hat\pi_n(W)} \right] = o_p(n^{-1/2}).$$ Since the left hand side is upper bounded by $$\|\hat\pi_n(W) - \pi_0(W)\| \|\hat{Q}_n(1,W) - Q_0(1,W)\|,$$ whenever sample positivity holds, we can see that we should fit the nuisance parameters by ensuring the estimates are close to the truth. Further, recall that minimizing closeness to a true expectation is equivalent to minimizing a prediction error. We can for example accomplish this by minimizing the cross validated error.

Although the TMLE is a different estimator, its influence function is the same and the same considerations apply. We can see this through the following motivation.

The superlearner is nicely described by Noah. It's purpose is to find nuisance parameters which ensure the above norms are as small as possible. As Noah explains, it does this by looking through several estimators and finding which to prefer. From above, we can see that the product of the rate of convergence of both estimators must be $$o_p(n^{-1/2})$$; since the superlearner inherits the best rate of convergence of its constituent estimators, it can be helpful here.

• Thank you for the nice technical but concise overview. I think your answer could be improved by specifying where and how SuperLearner fits into all this (i.e., making the remainder converge at the specified rate) since that seems to be OP's main concern.
– Noah
Commented Jan 10, 2023 at 22:13
• Hi Noah, I think your answer nicely describes the SuperLearner. I was only intending to explain why prediction error matters at all, addressing the second sentence in OP's first paragraph. I'll briefly add a few sentences that address your comment; let me know if they don't seem suitable.
– Ben
Commented Jan 10, 2023 at 22:22