I have a dataset with observations from about 50 countries and 20 years. My dependent variable is binary and I was wondering if I could use random forest to do out-of-sample predictions. My problem is: as far as I know, RF considers observations to be independent, which is not the case in my dataset. Is there any software (preferably an R package) that takes data structure into consideration when doing bootstrap sampling? I was thinking about something like GMERT, random effects combined with RF (see: How can I include random effects into a randomForest). However, due to my limited programming skills, I could not adapt the authors' code to use it with binary response variables. Any suggestions?

  • $\begingroup$ Time-series cross-section in the title plus how you describe your problem calls for a panel-data tag, and perhaps changing the title into something like "Random forest for panel data". Hopefully, that would help you attract the right people to answer your question. $\endgroup$ – Richard Hardy Jun 12 '15 at 8:14
  • $\begingroup$ Thanks, @RichardHardy! I've just edited the title as you suggested. $\endgroup$ – danilofreire Jun 13 '15 at 7:49
  • $\begingroup$ I am developing these methods and will keep you posted. $\endgroup$ – Randel Jul 28 '15 at 5:07
  • $\begingroup$ Thanks for your reply, @Randel. Please let me know when your code is ready and if I can help you with something. $\endgroup$ – danilofreire Jul 29 '15 at 5:36
  • $\begingroup$ Is there an analogous package in python? $\endgroup$ – user0 Jan 8 '17 at 18:34

I wrote a function for Mixed Logistic Random Forest for Binary Data. The usage is demonstrated with an example within the link. The prediction function is also available.

Following Hajjem's generalized mixed effects regression trees (GMERT), I used an EM-like algorithm and penalized quasi-likelihood (PQL) estimation.

  • The random forest part uses cforest() in the party package since it allows case weights.

  • The linearized mixed models are estimated with the lme4 package. So there are many possible options.

  • $\begingroup$ The function works great, @Randel! That's exactly what I needed. Thanks a lot! $\endgroup$ – danilofreire Jul 31 '15 at 9:13
  • $\begingroup$ @danilofreire great! If possible, I'd like to see the performance and comparison with other methods. We can discuss. :) $\endgroup$ – Randel Jul 31 '15 at 14:47
  • $\begingroup$ Cool, I'd love to help! Feel free to send me a message with any benchmark test or analysis you want me to run. Ah, and be sure that you'll get a big thank you in my future paper's acknowledgement section :) $\endgroup$ – danilofreire Aug 2 '15 at 3:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.