Use Mutual Information Feature Selection For Categorical Feature

Question

I have a dataset in which there are Features of both float and object type . I want to apply feature selection On this dataset in such a way that fisrt Find Mutual Information Score of all the features with Target. Then I choose 20 high score feature and do SFFS on them. So, I use mutual_info_classif in my codes but I get this error: could not convert string to float Because of one of my feature (Name=Grade) that is categorical and the unique value of this feature is :A,B,C,D. I have searched for finding the solution and everybody said that in this condition you should use one hot encoding. but I cant understand how to use one hot encoding? because I want to score each feature , not each category of feature. and If for example category=A get high score and category=D get Low score How can I decide to select or not select Grade feature?

Answer 1

You can use sklearn.preprocessing.OrdinalEncoder to convert your strings to integer values. One hot encoding would be the wrong choice since mutual information can directly work with categorical variables.

import pandas as pd
import numpy as np
from sklearn.feature_selection import mutual_info_classif as MIC
from sklearn.preprocessing import OrdinalEncoder


df = pd.DataFrame(
    {
        "grade": ["A", "B", "C", "D"],
        "number": [1.5, 2.5, 1.5, 2.5],
    }
)
target = np.array(["Pass","Pass","Fail","Fail"])

enc = OrdinalEncoder()
df[["grade"]] = enc.fit_transform(df[["grade"]])

MIC(df, target, discrete_features=[True,False])

array([0.69314718, 0. ])