1
$\begingroup$

i wanted to build a prediction model. Since my data had some missing data, I imputed data with the MICE algorithm. After that I wanted to do a regression with Random Forest.

Now I'm kinda stuck because:

I wanted to do Multiple Imputation with MICE because I wanted to show consideration for the variance of the missing variables in my model. So I imputed 5 data sets with MICE.

If i wanted to do a glm, I would build 5 models(for each imputed data set) and then pool them together. (Meaning in the end i have 1 model and the variance of my parameters will be higher)

Now what I want to do is to build a random forest. But I just can't find any strategies for this. Since RF doesn't have parameter estimations, I can't pool them together...

Has anyone worked on this before? or any advices what I should do?

Best wishes and thank you in advance! I really appreciate any help and answers

Ching

$\endgroup$
  • $\begingroup$ Maybe you can pool the predictions? $\endgroup$ – Michael M Aug 6 '16 at 12:56
2
$\begingroup$

This is not a direct answer to your question, and I don't have enough reputation to comment, but one thing you can do is use the Machine Learning in R package. There are many random forest learner implementations there that can use data with missing values. You can also tune the learners based on what your dataset is.

Links to the package and documentation are on the main tutorial page, here:

https://mlr-org.github.io/mlr-tutorial/release/html/index.html

Also, consider that answering your question becomes much easier if you provide a sample of your dataset.

If you need a direct answer, looping a series of RF calls on the imputed datasets might work. E.g. if you have five imputations:

res = data.frame(matrix(0,nrow=nrow(test),ncol=5)
for (i in 1:5){
  data = complete(miceResult, 1)
  rf.res = cforest(data,formula ~ [which formula?])
  res[,i] = predict(rf.res, test)
}

Then you can pool the results by majority voting or averaging, depending on your dataset. You can also group the 5 imputations together and train the learner with the combined dataset. Both methods are suboptimal, however.

Hope this helps.

$\endgroup$
  • $\begingroup$ Hi Mohammad! Thank you for your answer!! Appreciate your help and time!!! Yeah that's what i was planning too. Pooling the results. since I split my data into a training and a test dataset it's a bit more work: first create 5 data sets with mice for the training data and build a RF for each of them. Then for my test data, build 5 test data set. For each RF model i will predict my test datas and pool them. And of course for each Model. In the end I will pool 25 predictions. Also thanks for you references!!! $\endgroup$ – ching Aug 17 '16 at 16:00
  • $\begingroup$ @ching, you're welcome! Glad to have helped. If it answers your question you can mark the question answered. $\endgroup$ – Mohammad Ansarin Aug 18 '16 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.