Return to Answer

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use fewer trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithm like logistic regression, single decision tree, or naive Bayes classifier.

Alternatively, you can use model distillation, i.e. train a simpler model on the predictions of your model. It was shown that it can lead to retaining some of the performance of the original model.

Regarding the comment, the answer to Is the sum of two decision trees equivalent to a single decision tree? makes a theoretical argument that the weighted sum of many decision trees can be collapsed into a single tree. Let's assume it's doable to use it to collapse 100 trees into a single tree (I'm not totally convinced), in such a case you would have a 100-fold deeper tree than the individual trees in the initial model, this won't make the algorithm more time-performant! In a 100x deeper tree, you would make 100x more decisions on each split. If you used boosting, you would make the same number of decisions and then take a weighted sum of the results. In the end, you would only make it faster by omitting the weighted sum step that would have a negligible effect on the computation time.

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use fewer trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithm like logistic regression, single decision tree, or naive Bayes classifier.

Alternatively, you can use model distillation, i.e. train a simpler model on the predictions of your model. It was shown that it can lead to retaining some of the performance of the original model.

Regarding the comment, the answer to Is the sum of two decision trees equivalent to a single decision tree? makes a theoretical argument that the weighted sum of many decision trees can be collapsed into a single tree. Let's assume you collapse 100 trees into a single tree, in such a case you would have a 100-fold deeper tree than the individual trees in the initial model, this won't make the algorithm more time-performant! In a 100x deeper tree, you would make 100x more decisions on each split. If you used boosting, you would make the same number of decisions and then take a weighted sum of the results. In the end, you would only make it faster by omitting the weighted sum step that would have a negligible effect on the computation time. Moreover, since you would be basically concatenating the trees together, the final decision tree would be huge and you would need (exponentially!) more memory to store the tree (in the random forest or boosting, you could call the individual trees sequentially or in parallel).

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use fewer trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithm like logistic regression, single decision tree, or naive Bayes classifier.

Alternatively, you can use model distillation, i.e. train a simpler model on the predictions of your model. It was shown that it can lead to retaining some of the performance of the original model.

Regarding the comment, the answer to Is the sum of two decision trees equivalent to a single decision tree? makes a theoretical argument that the weighted sum of many decision trees can be collapsed into a single tree. Let's assume it's doable to use it to collapse 100 trees into a single tree (I'm not totally convinced), in such a case you would have a 100-fold deeper tree than the individual trees in the initial model, this won't make the algorithm more time-performant! In a 100x deeper tree, you would make 100x more decisions on each split, if you used boosting you would make the same number of decisions and then take a weighted sum of the results. In the end, you would only make it faster by omitting the weighted sum step that would have a negligible effect on the computation time.

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use fewer trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithm like logistic regression, single decision tree, or naive Bayes classifier.

Alternatively, you can use model distillation, i.e. train a simpler model on the predictions of your model. It was shown that it can lead to retaining some of the performance of the original model.

Regarding the comment, the answer to Is the sum of two decision trees equivalent to a single decision tree? makes a theoretical argument that the weighted sum of many decision trees can be collapsed into a single tree. Let's assume it's doable to use it to collapse 100 trees into a single tree (I'm not totally convinced), in such a case you would have a 100-fold deeper tree than the individual trees in the initial model, this won't make the algorithm more time-performant! In a 100x deeper tree, you would make 100x more decisions on each split. If you used boosting, you would make the same number of decisions and then take a weighted sum of the results. In the end, you would only make it faster by omitting the weighted sum step that would have a negligible effect on the computation time.

If it was possible to collapse multiple trees into a single decision tree, without loss of performance, the same effect should be achievable with a single decision tree. The trees constructed by boosting algorithm take different paths, i.e. choose different variables to make splits, make different splits, etc, there's no way to losslessly combine them into a single tree.

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use less trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithms like logistic regression, single decision tree, or naive Bayes classifier.

Algorithms such as random forest or boosting are not very computationally demanding at prediction time. If they are for you maybe you should change the hyperparameters, for example, use trees of smaller depth, print them, use fewer trees/iterations. If this still doesn't help because you have some tight constraints, consider using a simpler algorithm like logistic regression, single decision tree, or naive Bayes classifier.

Alternatively, you can use model distillation, i.e. train a simpler model on the predictions of your model. It was shown that it can lead to retaining some of the performance of the original model.

Regarding the comment, the answer to Is the sum of two decision trees equivalent to a single decision tree? makes a theoretical argument that the weighted sum of many decision trees can be collapsed into a single tree. Let's assume it's doable to use it to collapse 100 trees into a single tree (I'm not totally convinced), in such a case you would have a 100-fold deeper tree than the individual trees in the initial model, this won't make the algorithm more time-performant! In a 100x deeper tree, you would make 100x more decisions on each split, if you used boosting you would make the same number of decisions and then take a weighted sum of the results. In the end, you would only make it faster by omitting the weighted sum step that would have a negligible effect on the computation time.