Building a Regression Tree with one Gaussian Mixture Model at each node I am trying to build a regression tree that outputs both a mean and a covariance matrix for each leaf of the tree.
Ideally I would be able to have a Gaussian Mixture Model at each leaf. A first literature search was not productive. Has anybody seen this modeling structure before?
 A: Here is a thought how one could go about this: Structural Equation Model Trees are model-based trees with parametric multivariate normal distributions as outcomes in each leaf. There is an R-implementation called semtree based on OpenMx (http://openmx.psyc.virginia.edu) model specifications. The package is freely available here: http://brandmaier.de/semtree/.
OpenMx has tutorials on how to implement mixture models. So, in principle, it should work to specify a mixture model in OpenMx and pass it to the semtree()-function for recursive partitioning.
Here is another thought: When we assume that there is sample heterogeneity w.r.t. the original model, then both methods are addressing this issue differently. Trees recursively split the sample to find homogeneous subgroups based on observed predictors of heterogeneity whereas mixture models provide a probabilistic approach to estimate latent subgroups. Wouldn't it make sense to first run a tree to reduce sample heterogeneity and, in a second step, run mixture models in each leaf of the final tree to uncover the remaining heterogeneity that was not "explained away" in the first step? 
And here are some references to SEM trees and forests:
Brandmaier, A. M., Prindle, J. J., McArdle, J. J., & Lindenberger, U. (in press). Theory-guided exploration with structural equation model forests. Psychological Methods.
Brandmaier, A. M., von Oertzen, T., McArdle, J. J., & Lindenberger, U. (2014). Exploratory data mining with structural equation model trees. In J. J. McArdle & G. Ritschard (Eds.), Contemporary issues in exploratory data mining in the behavioral sciences (pp. 96-127). New York: Routledge.
Brandmaier, A. M., von Oertzen, T., McArdle, J. J., & Lindenberger, U. (2013). Structural equation model trees. Psychological Methods, 18, 71-86. doi: 10.1037/a0030001
