6
$\begingroup$

I'm modelling habitat selection using boosted regression trees (BRTs), which I prefer over linear models for a variety of reasons (modeling complex nonlinear relationships and interactions, multicollinearity, etc.). I now have a dataset that includes repeated measures for individual animals (100s or 1000s per individual). In a linear modeling framework, this is easily analyzed with random effects, but I haven't found any way to model clustered data with BRTs.

I found methods that produce individual regression trees (e.g., Mixed Effect Regression Trees (MERT), Regression Trees with Random Effects (REEMtree)) but the only mention I've found of random effects with BRTs is in Buston and Elith 2011, who did a post-hoc residuals analysis to identify group effects. Is there any way to account for clustering within the BRT analysis itself?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Are you aware of Mixed-effects random forest? $\endgroup$
    – Randel
    Commented Jul 27, 2015 at 20:26
  • $\begingroup$ Thanks Randel. Yes, I read that paper and tried contacting the first author for the R program, but got an auto-reply saying she's on maternity leave. Can you or anyone else point me to a copy of the R code? $\endgroup$
    – Michel
    Commented Jul 27, 2015 at 21:20
  • 1
    $\begingroup$ I am not sure if I have the right to share their code. But we wrote a simplified function MixRF. It's based on lme4 so there are many possibilities. Maybe you can try the example therein. $\endgroup$
    – Randel
    Commented Jul 28, 2015 at 4:29
  • $\begingroup$ We wrote a open-source Python implementation of mixed effects random forests from the paper quoted above. You can easily extend to make the base learner a boosted regression tree if you want. towardsdatascience.com/… github.com/manifoldai/merf $\endgroup$
    – Sourav Dey
    Commented Jun 24, 2018 at 3:23
  • $\begingroup$ Any update on this issue? I'm having the same problem four years later. @Michel how did you solve your problem? $\endgroup$
    – JMarcelino
    Commented Feb 27, 2020 at 11:43

1 Answer 1

Reset to default
1
$\begingroup$

If you want to use boosted trees instead of linear models, you essentially have three options: (i) you can ignore the grouping structure, (ii) you can include the grouping structure (i.e. the animal ID in your case) as a categorical variable (this corresponds to using fixed effects), (iii) or you can use a boosting algorithm that explicitly accounts for the clustering structure using random effects.

Option (i) is in general not a good idea as you ignore potentially important information when ignoring the clustering. Also, option (ii), including the grouping ID (animal ID in your case) as a categorical variable, can have drawbacks: learning is not efficient when there are many categories and a relatively small number of samples per category, and tree algorithms can have problems with high-cardinality categorical variables. The third option avoids these issues.

The GPBoost library with Python and R packages builds on LightGBM and allows for combining tree-boosting and mixed effects models. Simply speaking it is an extension of linear mixed effects models where the fixed-effects are learned using tree-boosting. See this blog post and Sigrist (2020) for further information.

Disclaimer: I am the author of the GPBoost library.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.