I have a dataset in which there are Features of both float and object type . I want to apply feature selection On this dataset in such a way that fisrt Find Mutual Information Score of all the features with Target. Then I choose 20 high score feature and do SFFS on them. So, I use mutual_info_classif in my codes but I get this error: could not convert string to float Because of one of my feature (Name=Grade) that is categorical and the unique value of this feature is :A,B,C,D. I have searched for finding the solution and everybody said that in this condition you should use one hot encoding. but I cant understand how to use one hot encoding? because I want to score each feature , not each category of feature. and If for example category=A get high score and category=D get Low score How can I decide to select or not select Grade feature?


1 Answer 1


You can use sklearn.preprocessing.OrdinalEncoder to convert your strings to integer values. One hot encoding would be the wrong choice since mutual information can directly work with categorical variables.

import pandas as pd
import numpy as np
from sklearn.feature_selection import mutual_info_classif as MIC
from sklearn.preprocessing import OrdinalEncoder

df = pd.DataFrame(
        "grade": ["A", "B", "C", "D"],
        "number": [1.5, 2.5, 1.5, 2.5],
target = np.array(["Pass","Pass","Fail","Fail"])

enc = OrdinalEncoder()
df[["grade"]] = enc.fit_transform(df[["grade"]])

MIC(df, target, discrete_features=[True,False])

array([0.69314718, 0. ])

  • $\begingroup$ Thank You , If the data is not ordinal, and is just categorical, can i use label encoder ? @Turamarth $\endgroup$
    – nasim_bb
    Feb 6, 2023 at 7:24
  • $\begingroup$ @nasim_bb yes. Practically speaking both have the same functionality. It doesn't matter which one you use. $\endgroup$
    – Turamarth
    Feb 6, 2023 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.