6
$\begingroup$

Is it right to use rfImpute to impute missing feature values on the whole data set and then use other regression/classification techniques on the new data set created? Or, is an rfImpute model intended to be fitted on a subset of the data and then that fitted model is somehow used to fill in missing values in the rest of the data?

To be clear, I only bring up these questions because rfImpute seems to require arguments for both X (features) and y (target variable). In addition, y cannot have any missing values. Does this mean that y gets used for the imputation of the features? Wouldn't this be harmful later when trying to fit models to the new imputed data set? Obviously the y values we are trying to predict in the future won't be known, so how could rfImpute fit into a machine learning pipeline?

Link: http://math.furman.edu/~dcs/courses/math47/R/library/randomForest/html/rfImpute.html

Thank you!

Improve this question
$\endgroup$
2
  • $\begingroup$ Are you trying to get 1 imputed dataset, or multiple imputed datasets (mice)? $\endgroup$
    – RNB
    Commented Aug 2, 2016 at 6:34
  • $\begingroup$ In either case, when trying to predict on new data in the future rfImpute would not work since those y values would be missing. It looks like there is a workaround: stackoverflow.com/questions/20537186/… However, I am still worried about the target variable being used to impute missing training data. $\endgroup$
    – Doug7
    Commented Aug 2, 2016 at 7:48

1 Answer 1

Reset to default
5
$\begingroup$

I'm not entirely sure if this is an answer to your question, but maybe you'll find it useful.

Maybe the author of the randomForest package would disagree with me, but I feel like the rfImpute() function is mostly used or called upon other imputation packages in their algorithms to impute many variables. If you only have one variable with missing data, then using this function as a stand alone may work. However, I think it is the case for most people that they have many variables with missing data in a datset that they'd like to impute. Enter the packages missForest and mice.

If you use the R package missForest, you can impute your entire dataset (many variables of different types may be missing) with one command missForest(). If I recall correctly, this function draws on the rfImpute() function from the randomForest package. For some reason (maybe others can elaborate), when you use the missForest() function, the other variables that are used to predict a single variable can also have missingness. So I think using this function and package are a nice idea if you are hoping to only get one dataset out, after all variables have been imputed.

The downside to using missForest() is that you only get one dataset, which does not allow you to take into account the uncertainty of your estimates (in your follow-on analytical models). So your analytical models will have incorrect confidence intervals if you just base the analysis on that one imputed dataset. If that doesn't matter to you, then I highly recommend this package and function, because it is very easy to use and specify your imputation model.

However, if you do need to get appropriate confidence intervals and pooled estimates in your analytical models, then you should probably use multivariate imputation by chained equations (MICE) approaches to imputation. For this, you can use the mice package. There is recent functionality within this package that allows you to specify which variables you'd like to impute with a random forest algorithm, and which you would like to use the usual methods (e.g. pmm). When specifying your imputation model with the mice() function, under methods you would do something like meth <- c("rfcat", "rfcont").

missForest has a nice vignette you can look up in R.

Here is a nice resource for how to set up your imputation models using mice:

http://www.stefvanbuuren.nl/publications/MICE%20in%20R%20-%20Draft.pdf

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for the useful information. missForest is well featured and works great. By the way, it is possible to get multiple datasets out of it. All you need to do is run it multiple times with different random seeds. $\endgroup$
    – Doug7
    Commented Aug 5, 2016 at 4:29
  • $\begingroup$ Good tip. I suppose if you impute, say 50, datasets that way, then you could theoretically do a bit of manual manipulation to get those datasets into a mids object so that you can use mice post-impuation pooling functions for your analytical models. $\endgroup$
    – RNB
    Commented Aug 5, 2016 at 10:02
  • $\begingroup$ Thanks for pointing out missForest (I hadn't seen it before), but it doesn't use rfImpute. rfImpute will populate the missing values with the vector median (if continuous) or mode (if factor). missForest seems to actually use a random forest algorithm in order to impute the values. I think that's what OP was looking for anyway, but I wanted to clarify. $\endgroup$
    – EconomySizeAl
    Commented May 8, 2018 at 15:02
  • $\begingroup$ Regarding the comment "rfImpute will populate the missing values with the vector median (if continuous) or mode (if factor)". This is not quite correct. It will use the medians to initialize and then it iteratively updates these values based on the randomForest proximities. $\endgroup$
    – Danny
    Commented Feb 3, 2022 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.