I have a dataset that contains mostly categorical features.

feature_1 = ['House', 'Car', 'Idea', 'House']

For algorithms like extreme gradient boosting (XGBoost) or also most random forest implementations it was practice to convert these features into multiple binary features like this:

feature_1_House = [1, 0, 0, 1]
feature_1_Car = [0, 1, 0, 0]
feature_1_Idea = [0, 0, 1, 0]

So, these features can be treated as normal numerical features using split values (for random forest etc). The problem with this is that it blows up the dataset. Imagine you have one feature with 100 categories. These feature would be converted to 100 features not unlikely that most of the values are zero. I wonder which implementations nowadays can handle categorical data internally without need to convert the categorical features like above. I think there were plans for XGBoost to implement this. And how would you handle this with neural networks. Would you rather generate a sparse multi-dimensional input-vector?

I know the question might be a bit unspecific, but I think, it is a viable question that comes up frequently.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.