Just wonder if there is any caveat if one fits regular regression trees to clustered data but ignores the clustered structure of the data. More generally, how bad it would be if we fit regression trees to data in which the observations are not independent from each other (in terms of prediction accuracy, interpretation of the results...etc)?

We know that tree-based models are non-parametric, and there is no such thing as "violating the parametric assumption", which is the rationale for developing all the fancy models instead of just using OLS when the data is longitudinal or clustered. However, all the examples that I have come across when learning tree-based methods are based on cross-sectional data. I haven't come across any discussions on the potential consequences (in terms of prediction accuracy, ...etc.) if we ignore the clustered nature of the data. Any insight on this?

Several methods have been proposed to integrate tree-based models to mixed-effects regression models. However, it appeared to me that these methods were from a more regression-diagnostic point of view (e.g., Sela and Simonoff, 2012), and the scope and view may be different from what i am asking here. Their motivation was to use regress tree to account for the residuals; the tree was implemented after fitting a (linear) mixed-effects model. In other words, they first imposed a parametric structure to the data, fit the regression model, apply tree-based methods to analyze the residuals, and use the results from tree-based methods to improve the regression model estimates.

Finally, a bit background on my data: The response is a positive and continuous variable. There are about 350 clusters, each of which has 3 measurements. Thus there are 350x4 =1400 responses (balanced panel). There are about 20 predictors. The goal is to select the predictors that are strongly associated with the response.

  • 1
    $\begingroup$ For me, Sela and Simonoff (2012) used trees to estimate the fixed-effects part and mixed-models to account for the clustering in the residuals. $\endgroup$ – Randel Nov 23 '15 at 21:36
  • $\begingroup$ Just came across a paper by Bel and Allard (2009) that modifies the CART algorithm for clustered samples, which is common in ecological and environmental studies. Still, before adapting their algorithms, it would be great to know whether the CART is "good enough" or not for the current dataset which is a balanced panel. I do not have a good answer for this yet. $\endgroup$ – TCL Nov 24 '15 at 15:33
  • $\begingroup$ Then why not start from random forest? $\endgroup$ – Randel Nov 24 '15 at 16:40
  • $\begingroup$ Is random forest good at dealing with clustered data? $\endgroup$ – TCL Nov 24 '15 at 16:48
  • 1
    $\begingroup$ See this. $\endgroup$ – Randel Nov 24 '15 at 16:53

As pointed out by @Randel in the comments, the RE-EM tree approach by Sela and Simonoff (2012, Machine Learning, 86(2), 169-207, doi:10.1007/s10994-011-5258-3) and the MERT approach by Hajjem, Bellavance, Larocque (2011, Statistics and Probability Letters, 81, 451-459, doi:10.1016/j.spl.2010.12.003) are rather similar. Both iterate between adjusting for random effects (= correlated observations) and fitting a regression tree like CART. Thus, the result is a regression tree as you get from CART (i.e., with just a predicted constant in each leaf of the tree) but adjusted for a global random effect (e.g., a random intercept for a cluster to adjust for within-cluster correlations).

Therefore, the original motivation of these trees (and corresponding forests) is really to obtain regression trees for clustered data. However, they can also be used for regression diagnostics as pointed out by Simonoff (2013, Statistical Modelling, 13(5-6), 459-480, doi:10.1177/1471082X13494612).

We recently proposed an extension of RE-EM tree/MERT that can also include more complicated models in each node of the tree (e.g., a treatment effect) and is applicable not only to continuous responses but also binary or cout data (Fokkema et al., 2015).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.