I apply (boosted) regression trees to build predicitive models with continuous outcome (
gbm). While regression trees (rt) split the feature space and predict a constant in each terminal leafe a model tree fits a model in each leafe.
Without too much experience here I would assume that non-boosted trees perform rather bad (too few flexibility) while I know (after some applications) that their boosted versions work well (adding up all those weak learnes splitting with different features and different data we can approximate the target).
I thought about trying out model trees (using
partykit as here). What can I expect? Do practitioners use model trees? Is the danger of a large generalization error bigger than with regression trees?
Thanks for sharing your experience!