Why symmetric trees are not used in xgboost

Question

I was reading about catboost and found out that catboost uses symmetric trees. The reasons for using symmetric trees is: The choice of oblivious trees has several advantages compared to the classic ones: • Simple fitting scheme • Efficient to implement on CPU • Ability to make very fast model appliers • This tree structure works as a regularization, so it can provide quality benefits for many tasks

If symmetric trees has so many advantages, why xgboost does not use symmetric trees as well? Is it because of the fact that when xgboost was released symmetric trees were not as prevalent?

Answer 1

Mostly yes. Simple logistics:

1. Symmetric trees were not around when the original XGBoost implementation came out.
2. Nobody using XGBoost is bothered enough to code them as the current performance is deemed adequate.
3. New base-learners are no clear wins. Initially, people thought for example that DART (Dropouts) was also a game-changer and it turned out to be... OKish...?

Just to be clear, symmetric trees are not guaranteed to be better. If anything, we empirically know that CatBoost has not dominated any of the other implementations, it is as good as. Also given that we have so many different ways to regularise a GBM already, the ability to have one more regularisation lever is not so groundbreaking.

Symmetric trees have a clear advantage when it comes to inference time but for GBM applications, the inference speed usually is a small aspect of the project. Contrary to that, Catboost's grandparent from Yandex, MatrixNet, was a GBM specifically geared towards recommender systems where inference time is crucial, and during a time when GPUs were not so mature; this has carried forward. In contrast, XGBoost, which started as part of the DCML, has moved towards high-performance computing on the GPU work first by cuBLAS and nowadays via Rapids, a far more GPU-first approach. (Side-note: using a GPU doesn't equate to HPC.)