I have a dataset with observations from about 50 countries and 20 years. My dependent variable is binary and I was wondering if I could use random forest to do out-of-sample predictions. My problem is: as far as I know, RF considers observations to be independent, which is not the case in my dataset. Is there any software (preferably an R package) that takes data structure into consideration when doing bootstrap sampling? I was thinking about something like GMERT, random effects combined with RF (see: How can I include random effects into a randomForest). However, due to my limited programming skills, I could not adapt the authors' code to use it with binary response variables. Any suggestions?

  • $\begingroup$ Time-series cross-section in the title plus how you describe your problem calls for a panel-data tag, and perhaps changing the title into something like "Random forest for panel data". Hopefully, that would help you attract the right people to answer your question. $\endgroup$ Jun 12, 2015 at 8:14
  • $\begingroup$ Thanks, @RichardHardy! I've just edited the title as you suggested. $\endgroup$ Jun 13, 2015 at 7:49
  • $\begingroup$ I am developing these methods and will keep you posted. $\endgroup$
    – Randel
    Jul 28, 2015 at 5:07
  • $\begingroup$ Thanks for your reply, @Randel. Please let me know when your code is ready and if I can help you with something. $\endgroup$ Jul 29, 2015 at 5:36
  • $\begingroup$ Is there an analogous package in python? $\endgroup$
    – sjw
    Jan 8, 2017 at 18:34

1 Answer 1


I wrote a function for Mixed Logistic Random Forest for Binary Data. The usage is demonstrated with an example within the link. The prediction function is also available.

Following Hajjem's generalized mixed effects regression trees (GMERT), I used an EM-like algorithm and penalized quasi-likelihood (PQL) estimation.

  • The random forest part uses cforest() in the party package since it allows case weights.

  • The linearized mixed models are estimated with the lme4 package. So there are many possible options.

  • $\begingroup$ The function works great, @Randel! That's exactly what I needed. Thanks a lot! $\endgroup$ Jul 31, 2015 at 9:13
  • $\begingroup$ @danilofreire great! If possible, I'd like to see the performance and comparison with other methods. We can discuss. :) $\endgroup$
    – Randel
    Jul 31, 2015 at 14:47
  • $\begingroup$ Cool, I'd love to help! Feel free to send me a message with any benchmark test or analysis you want me to run. Ah, and be sure that you'll get a big thank you in my future paper's acknowledgement section :) $\endgroup$ Aug 2, 2015 at 3:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.