Use the target variable during imputation?

Question

There is a quite old yet very good question about the proper way for using rfImpute but to me the question raised by Doug7 (whether the target variable y gets used for the imputation of the features Xi and whether that would be harmful later when trying to fit models to the new imputed data set) has not really been answered.

The accepted answer by RNB points towards using mice or missForest R packages for imputation instead of randomForest's rfImpute but the differences are that these packages

1. build individual imputation models for each variable Xi while rfImpute builds a single model for the whole dataset
2. you can choose to ex- or include the target variable y as a variable for all these individual imputation models in mice and missForest but you must include y in rfImpute

So I would like to raise the question about whether or not to use y during impute again but more generally and not limited to the use of rfImpute. The only study I could find about this is a 2017 study which focuses more on whether or not to also impute missing values of y but it also has some experiments with the potential to shed light on the question of whether or not to make use of y during imputation. Since I think there is an error for item #3 in the table listing the experiments they made let me list the first 3 experiments relevant to my question again here as I understand it:

1. complete case analysis as a reference (no multiple imputation [mi])
2. no imputation of missing y, y not used in mi model
3. no imputation of missing y, y used in mi model

So in a nut shell, the question is: can you use strategy #3 from above (which would allow you using the computationally much cheaper rfImpute) or should you never make use of y in an imputation model (which would force you to go with something a lot more expensive like mice or missForest) and go with #2 instead when your concern is a model fit on the imputed data set?

My gut feeling told me not to use #3 but the study seems to show that when it comes to bias #3 clearly outperforms #2 and when it comes to error at least for larger data sets the same seems to be true. But this refers to the quality of the impute, not to the impact onto models fit on the new imputed data set!

What are your thoughts on this?

Thanks, Mark

Answer 1

It's seldom good to argue "always" or "never" in response to this type of question. Much depends on the details of the data, your understanding of the subject matter, and what tradeoffs you are willing to make. What follows are some links to guidance that you can apply as needed.

Stef van Buuren's Flexible Imputation of Missing Data (FIMD) is a reliable reference. In general, the book doesn't even distinguish "outcomes" from "predictors" in terms of imputation. All data are typically lumped together into a single matrix with indicators of which elements are missing in the original data.

The missing-at-random (MAR) assumption underlying multiple imputation is that missingness can depend on observed but not unobserved information. Thus it makes sense to use as much of the observed information as reasonable in multiple imputation. I don't see any reason completely to ignore whatever information the observed $Y$ values might provide for imputation, as in your Strategy 2. You might choose for some reason not to, based on your understanding of the subject matter, but that should be a conscious choice.

Even if you choose not to use the outcome $Y$ in the imputation, it's still generally important to perform multiple imputations, fit models on each of the imputed data sets, and then combine the results as explained in Section 5. A quote from Rubin at the start of Chapter 2 puts it simply:

Imputing one value for a missing datum cannot be correct in general, because we don’t know what value to impute with certainty (if we did, it wouldn’t be missing).

As the page you link notes, confidence intervals and the like won't be correct if you only use a single imputation.

Section 2.7 of FIMD discusses some potential exceptions relating to this question.

First, your Strategy 3 never lets you use cases with a missing outcome value $Y$. That's OK if the only missing data are in $Y$. But that's not the situation that you are describing. Otherwise you run into the problems with complete-case analysis discussed in that Section and elsewhere:

The efficiency of complete-case analysis declines if $X$ contains missing values, which may result in inflated type II error rates.

Second, complete-case analysis is OK, in terms of bias, if the probability of missingness is independent of the outcome $Y$ in a regression model against predictors $X$:

The first special case occurs if the probability to be missing does not depend on $Y$. Under the assumption that the complete-data model is correct, the regression coefficients are free of bias...

That might argue for Strategy 1 if you can make that assumption about missingness! In that context, I think about single imputation with rfImpute() as extending complete-case analysis to cases with some missing $X$ values but observed $Y$ values that might otherwise be omitted. Insofar as the $Y$ values contain information about the values of the missing $X$ entries in the data matrix, that should tend to increase power (although it will again introduce problems with things like confidence intervals).