Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-based algorithm (that we also use as black box predictors). Preference for a collection of explicitly defined tree leaves over an unobservable ensemble of many such leaves (i.e. a tree-based model) is motivated by explainability and traditional century-old insurance industry methods of discovering data clusterings with higher average target values (typically some risk measures).
The main difficulty is how to use data partitioning information from more than a single tree, utilizing benefits of ensembling and cross-validation. Note that in practice tree structure tends to differ between all trees in the ensemble (such as boosting rounds in a GBDT algo) and between all cross-validation folds, so a leaf data usually cannot be averaged from multiple trees, because every leaf in every tree by default is unique.
One approach would be such a set of hyperparameters that would ensure fixed tree structure for all trees in the ensemble, another: re-using the same pre-trained tree-based model object across all cross-validation folds or members of an ensemble, and yet another - a modified
predict function that would keep a tree structure intact (ideally also with unchanged split points).