Including the outcome variable in multiple imputation

Question

I'm trying to perform binary classification on a dataset with missing values. I used sklearn's iterative imputer to impute these values and I got pretty good results. However, I realized that I was performing the imputation on a dataframe that contained the output variable. This seems wrong to me, but when I googled it I found several sources that said the outcome variable should be included. I would like to believe this because my model performs much better when I do the imputation with the outcome variable, but it doesn't make sense to me. If I include the outcome variable how am I supposed to apply this imputation to new data which doesn't have the outcome variable yet?

Answer 1

It's a simple result of Bayes rule. If the outcome variable is conditionally dependent on any of the inputs, then the imputation model for the inputs will not be correct unless it includes the outcome as a feature.

To impute missing data in a new data frame, you have to train an imputation model which includes missing outcome as a missingness pattern. In that case, supposing you are using MICE as it's one of the most widely implemented multiple imputation platforms out there, you can impute the whole dataset, including the outcome, and use that information to perform prediction, simulation, or any number of tasks. However, once the actual outcome is observed, you should replace the "missing" column of outcomes and redo the imputations (because you now observed the outcome), that way you get efficient inference.

Or you can omit the outcome from imputation. This marginal approach will increase the MSE of predictions by incorporating some bias, and less efficient inference, but that in and of itself is not a reason to exclude a method if the actual scientific question implies it.