Is it common to have a different regression technique at the leaves of a regression tree (for instance linear regression)? I've been searching for it for the past hour but all I find are implementations that have a constant value at the trees' leafs. Is there a reason why this is/is not common?
I think it isn't more popular is that a lot of the robustness of ensembles of decision tree style models comes from the fact they allways predict constant values in range they've seen.
Outliers in the data generally just get lumped together with the highest/lowest normal values in the data on the last leaf and don't cause strange predictions or throw off coeffichents.
They also don't suffer from issues with multicolinearity as much as linear models.
You mighe be able to adress these issues in an implementation but its probally easier and more robust to just add more trees in an ensemble via boosting or bagging untll you get the smoothness you need.
There has been quite some research on this topic over the last decades, starting with the pioneering efforts of Ciampi, followed by Loh's GUIDE, and then also Gama's functional trees or the model-based recursive partitioning approach by us. A nice overview is given in @Momo's answer to this question: Advantage of GLMs in terminal nodes of a regression tree?
Corresponding software is less widely used than simple constant-fit trees as you observe. Part of the reason for this is presumably that it is more difficult to write - but also more difficult to use. It just requires more specifications than a simple CART model. But software is available (as previously pointed out here by @marqram or @Momo at: Regression tree algorithm with linear regression models in each leaf). Prominent software packages include:
In the Weka suite there are
M5P(M5') for continuous responses,
LMT(logistic model trees) for binary responses, and
FT(functional trees) for categorical responses. See http://www.cs.waikato.ac.nz/~ml/weka/ for more details. The former two functions are also easily interfaced through the R package
Loh's GUIDE implementation is available in binary form at no cost (but without source code) from http://www.stat.wisc.edu/~loh/guide.html. It allows to modify the details of the method by a wide range of control options.
Our MOB (MOdel-Based recursive partitioning) algorithm is available in the R package
partykit(successor to the
mob()function gives you a general framework, allowing you to specify new models that can be easily fitted in the nodes/leaves of the tree. Convenience interfaces
glm()are directly available and illustrated in
vignette("mob", package = "partykit"). But other plugins can also be defined. For example, in https://stackoverflow.com/questions/37037445/using-mob-trees-partykit-package-with-nls-model
mob()is combined with
nls(). But there are also "mobsters" for various psychometric models (in
psychotree) and for beta regression (in
I found a method that does just this (a decision tree, where the leafs contain a linear-regression instead of an average value). They are called model trees  and an example is the M5P algorithm of weka. In M5P a linear regression is at each leaf.
Edit: I found another package/model that does something similar and seems to give very good results for my datasets: cubist. An implementation in R is given by the cubist package . Cubist adds boosting ensembling to M5P and what it calls 'instance based corrections'.
: Torgo, L. Functional models for regression tree leaves. In Proceedings of the 14th International Conference on Machine Learning, pp. 385–393. Morgan Kaufmann, 1997.
: Cubist model Cubist: Rule- And Instance-Based Regression Modeling https://cran.r-project.org/web/packages/Cubist/index.html