I want to use the R
missForest() function at work to perform missing value imputation. However, after reading up on the algorithm more, I can't decide how to impute future/test data - running the algorithm on just the
test data, or run it on
I have heard many times that you training & test datasets should not inform one another at all, but this algorithm is an example of where I think that doing so would increase predictive performance a lot, and crucially I see no reason why I wouldn't be able to apply this same approach in production for future unseen data.
Here's an example of what i'm proposing. First I do a train/test split and introduce missing values:
library(datasets) library(missForest) library(randomForest) data("iris") # train/test: set.seed(2) train_index <- sample(nrow(iris), 120) feature_train <- as.matrix(iris[train_index, 1:4]) feature_test <- as.matrix(iris[-train_index, 1:4]) # introduce NA's to both: feature_train[sample(length(feature_train), 40)] <- NA feature_test[sample(length(feature_test), 10)] <- NA
I impute the training data and build a model on the imputed data:
imp_feature_train <- missForest(feature_train)$ximp rf <- randomForest(x = imp_feature_train, y = iris$Species[train_index])
For the test data, this is the method that I prefer. It uses the imputed training data to inform
missForest() in imputing the test data. Is there any issue with this, as long as (if the model was put into production) I also used this training data to inform any future data?
feature_train_test <- rbind(feature_test, imp_feature_train) imp_feature_test_v1 <- missForest(feature_train_test)$ximp[1:nrow(feature_test), ] pred_test_v1 <- predict(rf, imp_feature_test_v1) sum(pred_test_v1 == iris$Species[-train_index]) # 27
The reason I want to use this approach: imagine your training dataset is 1M observations, and you are building a model that must be ran hourly on 1k observations. The 'quality' of the missing value imputations will surely be higher if you use those 1M observations to 'inform' the
missForest() algorithm, as apposed to simply running the algorithm on the 1k observations alone.
This imputes the train and test datasets completely separately. I imagine that performance would degrade considerably as the test dataset gets smaller:
imp_feature_test_v2 <- missForest(feature_test)$ximp pred_test_v2 <- predict(rf, imp_feature_test_v2) sum(pred_test_v2 == iris$Species[-train_index]) # 26
Thanks in advance.