SelectKBest for different column types: categorical and continous features

Question

I have a dataset of thousands of continuous variables and a few categorical ones. If I use SelectKBest with mutual_info_classif(X, y) then the categorical ones are discarded since they are categorical variables and this function does not work well with them. I know that the categorical ones are important and I want to perform feature selection through SelectKBest only with the continuous ones. I would like to include this in an sklearn pipeline.

How can I tell the sklearn pipeline to perform KBest only on certain columns (the continuous ones in this case)?. Do we always have to select our features in different ways depending on their nature (continous, categorical, etc...)?

Answer 1

Use ColumnTransformer, applying SelectKBest only to the continuous variables. You can apply some other feature selection/transformation to the categorical variables, or set remainder=passthrough to send the categorical features through untouched (aside from the order of the columns).

To the broader question, some feature selection methods should be applicable to different feature types (model-based wrapper methods come to mind). But feature selection is particularly tricky, so experimenting with your data is probably best.