3
$\begingroup$

I have several numeric features and two catigorical features. The catigorical features have no order (nominal), they are for example colors; red, blue, yellow, green... and place of birth; USA, Spain, Germany. The target feature is binary catigorical which can be easily converted to numeric with 1 and 0.

How can XGBoost handle those catigorical features? should I switch from catigorical to numeric?

If so, how? I mean, can red, blue, yellow.. become 1, 2, 3 ? will that affect the information provided by that feature column in any way?

$\endgroup$

1 Answer 1

3
$\begingroup$

Yes, XGBoost handles categorical features nowadays (from ver. 1.5.x onwards).

The functionality is currenty experimental but as it stands XGBoost handles categorical features the same way that LightGBM does; in short, the categories are binned and then sorted according to the training objective at each split. Effectively that is a one-dimensional clustering and allows us to find the optimal split of this "binned" variable fast and efficiently. A worked example can be found under the Categorical Data tutorial in the XGBoost homepage.

As final comment: when using categorical features with relatively low cardinality (e.g. <10 categories) all implementations are pretty interchangeable; differences start materialising when we have higher cardinality features. Similarly, for very high cardinality features (1000+) like textual descriptions, embedding the categories in a low-dimensional numeric space might be simpler and easier to work with.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.