# (Low cardinality) categorical features handling in gradient boosting libraries

In some popular gradient boosting libraries (lgb, catboost), they all seems like can handle categorical inputs by just specifying the column names of the categorical features, and pass it into a fit or model instance by setting it to categorical_feature=.

(except in xgboost:

Categorical features not supported

Note that XGBoost does not support categorical features; if your data c ontains categorical features, load it as a NumPy array first and then perform one-hot encoding.)

When the cardinality is high, I can intuitively understand the advantages of employing some tricks of handling the cardinality (maybe like mean encoding) comparing to one-hot encoding, since one-hot would just result a huge sparse matrix, and it's probably difficult for the model to learn well without developing a deep tree.

My questions are:

1) Why xgboost does not handle categorical features natively? Is it because the advantages of doing all these manipulations in lgb and catboost is actually not significant in practice?

2) For lgb and catboost, when the cardinality in the categorical features is low, is it better to still pass the categorical columns as categorical_feature= or use one-hot encoding (since in this case there won't be a giant sparse matrix)?

1. The XGBoost implementation of GBM does not handle categorical features natively because it did not have to. The methodological breakthrough of XGBoost was the use of Hessian information. When other implementations (e.g. sklearn in Python, gbm in R) used just gradients, XGBoost used Hessian information when boosting. Simply put, it obliterated them in terms of speed. Handling categorical variables was an after-thought. LightGBM and CatBoost build on the work of XGBoost and primarily focus on the handling of categorical features and growing "smarter" trees. Especially for CatBoost, that is developed mainly by Yandex, an Internet search provider, the ability to work efficiently with very high cardinality features (e.g. query types) is crucial functionality.