role of onehotencoding in Feature selection

Question

I need to clarify about feature selection.

I am working on Kaggle breast cancer dataset (https://www.kaggle.com/datasets/reihanenamdari/breast-cancer). It is a categorical dataset. There are 15 features including one class level. So, to encode the categorical columns I need some encoding. I used labelencoder and onehotencoder. Now, if I go with label encoder I got the feature selection score for 15 features. However, if I go with onehotencoder I got feature selection for 39 features. I know why this was happened. Because onehotencoding transfer the dataset 0 1 format for all responses of a category.

Now the categorical dataset is a nominal categorical dataset. So, it is better to use onehotencoding rather than labelencoder.

code block for labelencoder and feature selection

fs = SelectKBest(score_func=chi2, k='all')
fs.fit(X_train, y_train)
X_train_fs = fs.transform(X_train)
X_test_fs = fs.transform(X_test)  

for i in range(len(fs.scores_)):
    print('Feature %d: %f' % (i, fs.scores_[i]))

Output for feature selection when using labelencoder

Feature 0: 18.397683
Feature 1: 0.907448
Feature 2: 1.948516
Feature 3: 61.664194
Feature 4: 270.739282
Feature 5: 290.261719
Feature 6: 4.682429
Feature 7: 12.392248
Feature 8: 0.668526
Feature 9: 921.736979
Feature 10: 7.633741
Feature 11: 19.629019
Feature 12: 20.830701
Feature 13: 1481.246548
Feature 14: 5428.386792

Code block for onehotencoding with feature selection

s = SelectKBest(score_func=chi2, k='all')
fs.fit(X_train, y_train)
X_train_fs = fs.transform(X_train)
X_test_fs = fs.transform(X_test)
for i in range(len(fs.scores_)):
    print('Feature %d: %f' % (i, fs.scores_[i]))

Output for onehotencoding and featureselection

Feature 0: 14.659096
Feature 1: 850.014288
Feature 2: 21.873525
Feature 3: 1485.277925
Feature 4: 5993.193209
Feature 5: 0.512006
Feature 6: 20.922138
Feature 7: 4.021212
Feature 8: 5.601705
Feature 9: 1.133983
Feature 10: 1.224111
Feature 11: 9.484409
Feature 12: 10.513141
Feature 13: 27.287016
Feature 14: 2.181016
Feature 15: 12.428941
Feature 16: 45.034507
Feature 17: 51.032404
Feature 18: 9.335082
Feature 19: 167.640306
Feature 20: 46.638949
Feature 21: 1.276569
Feature 22: 167.640306
Feature 23: 8.832149
Feature 24: 10.731865
Feature 25: 51.766011
Feature 26: 11.010064
Feature 27: 16.180921
Feature 28: 11.367710
Feature 29: 51.766011
Feature 30: 11.010064
Feature 31: 16.180921
Feature 32: 11.367710
Feature 33: 0.928991
Feature 34: 35.539577
Feature 35: 9.218108
Feature 36: 128.157306
Feature 37: 20.555553
Feature 38: 99.097837

My question is how could I use onehotencoding for feature selection?

Answer 1

OneHotEncoding is not used for feature selection. It is used for encoding the categorical variables. If you want to perform feature selection there are a lot of techniques out there.

Here is a list of some of them.

Here is an article which describes which technique to use when.

PS: OneHotEncoding is good for learning purpose but it is not the widely used technique when it comes to real world applications. There are other much better encoding techniques out there!

Here is a pip library which contains almost all encoding techniques. Simple install using pip and start using!

Cheers!