2
$\begingroup$

I have a dataset of thousands of continuous variables and a few categorical ones. If I use SelectKBest with mutual_info_classif(X, y) then the categorical ones are discarded since they are categorical variables and this function does not work well with them. I know that the categorical ones are important and I want to perform feature selection through SelectKBest only with the continuous ones. I would like to include this in an sklearn pipeline.

How can I tell the sklearn pipeline to perform KBest only on certain columns (the continuous ones in this case)?. Do we always have to select our features in different ways depending on their nature (continous, categorical, etc...)?

$\endgroup$
1
$\begingroup$

Use ColumnTransformer, applying SelectKBest only to the continuous variables. You can apply some other feature selection/transformation to the categorical variables, or set remainder=passthrough to send the categorical features through untouched (aside from the order of the columns).

To the broader question, some feature selection methods should be applicable to different feature types (model-based wrapper methods come to mind). But feature selection is particularly tricky, so experimenting with your data is probably best.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is what I needed. Not exactly, because I wanted to apply KBest only to numerical features and "passthrough" will leave them as they were. But with ColumnTransformer I can apply OneHotEncode to categorical variables and KBest to Numerical. $\endgroup$ – Brandon Feb 19 at 12:19
  • $\begingroup$ @Brandon, right, sorry, I had your situation backwards. I've edited the answer accordingly. $\endgroup$ – Ben Reiniger Feb 19 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.