I'm trying to determine what to do with categorical feature when using recursive feature selection.
I've looked around this forum and elsewhere and most discussions focus on one-hot-encoded features and whether or not to eliminate particular dummy variables. I don't think this exactly answers the question.
It seems from the preponderance of discussion categorical variables must be included/excluded in whole instead of in "parts" (e.g., some dummy variable categories but not others). That said, how do we handle 'whole' categorical variables in the context of the data set?
If I have labeled categorical variables, does it makes sense to encode them hierarchically (e.g.
LabelEncoder in Sklearn) and then run an RFE process?
Other discussion suggests holding out all categorical variables. That doesn't seem correct intuitively. If you perform RFE on continuous variables, get the best performing combination, and then 'lump in' the held out categoricals you're going to get different - and likely non-optimal - results.
Please share your thoughts.