I'm working on estimating the performance of a predictive model. However, a rather large amount of covariate data is missing. My initial idea was to use cross-validation to estimate the prediction error by doing the following in each step:
- Fit an imputation model on the training part
- Use the imputed training data to fit a predictive model
- Impute the test data using the imputation model from the first step
- Use the imputed test data to get an estimate of the prediction error.
When building the imputation model I would include the outcome so that the imputed values in the training set are not independent of the outcome. However, my gut-feeling tells me that using the outcome to impute the test data might lead to an optimistic estimate of the prediction error. I was wondering if this would be the standard approach to take or if there's some literature available that deals with this problem?