0
$\begingroup$

Does it make sense to apply recursive feature elimination on a feature set pre-processed with One-Hot Encoding?

This is my code for feature selection:

xgb = XGBClassifier(n_estimators=100, 
                    objective='multi:softprob', 
                    num_class=4, 
                    random_state=42)
rfecv = feature_selection.RFECV(estimator=xgb, 
                                step=10, 
                                cv=model_selection.StratifiedKFold(2), 
                                scoring='f1_weighted', 
                                n_jobs = -1,
                                verbose = 2)
rfecv.fit(X_train, y_train)

DataFrame X_train contains both continuous and categorical features. Categorical features are one-hot encoded, while continuous features are passed through MinMaxScaler.

I am not sure if it makes sense to eliminate one-hot encoded columns using RFECV. Maybe I should run RFECV on continuos features only? Or I should apply one-hot encoding somehow at each iteration of RFECV?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.