# How to fix the tree structure for a tree-based algorithm?

Background

Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-based algorithm (that we also use as black box predictors). Preference for a collection of explicitly defined tree leaves over an unobservable ensemble of many such leaves (i.e. a tree-based model) is motivated by explainability and traditional century-old insurance industry methods of discovering data clusterings with higher average target values (typically some risk measures).

Problems

The main difficulty is how to use data partitioning information from more than a single tree, utilizing benefits of ensembling and cross-validation. Note that in practice tree structure tends to differ between all trees in the ensemble (such as boosting rounds in a GBDT algo) and between all cross-validation folds, so a leaf data usually cannot be averaged from multiple trees, because every leaf in every tree by default is unique.

Options

One approach would be such a set of hyperparameters that would ensure fixed tree structure for all trees in the ensemble, another: re-using the same pre-trained tree-based model object across all cross-validation folds or members of an ensemble, and yet another - a modified predict function that would keep a tree structure intact (ideally also with unchanged split points).

• This will provide no benefit. The wikipedia article on ensemble learning theory says "Empirically, ensembles tend to yield better results when there is a significant diversity among the models." By specifying the tree structure you'll essentially be creating copies of the same model. Commented May 7, 2022 at 3:36
• @RyanVolpi yes, I realize it defeats the purpose of ensembling (by forgoing variance reduction), but there are other significant benefits from those tree-based models: 1) built-in cross-validation and 2) computational speed. I think the models should be forced to specialize to just 1 tree to get the above benefits without distorting predictions (in contrast to picking 1 tree from the ensemble, e.g. with lgbm.create_tree_digraph)? Commented May 7, 2022 at 7:21