I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then convert these trees to SAS which results in hundreds of thousands of lines of code. Many of the trees are very similar, however. This begs the question of whether subsets of related trees can be combined to reduce the amount of code that needs to be generated.
My first approach was to find trees that differed only by their final predictions and de-dupe them. These trees had identical interactions and splits at every node. This works well when the number of interactions is small (<3) however, there is virtually no performance gain when the interactions increase beyond this size as the trees are increasingly likely to be unique.
My next thought is that many of the first or second splits are going to be identical so why not consolidate that logic and nest the remaining nodes within? Before heading down that path, though, I thought I would reach out for guidance or insight here.
Is there a way to combine decision trees output from a GBM process to reduce the number necessary to calculate the final score?