XGBoost will produce different values for feature importances with different hyperparameters on the same dataset. When using XGBoost as a feature selection algorithm for a different model, should I therefore optimize the hyperparameters first? Or there are no hard and fast rules, and in practice I should try say both the default and the optimized set of hyperparameters and see what really works?
2
-
3$\begingroup$ You shouldnt use xgboost as a feature selection algorithm for a different model. Different models use different features in different ways. Theres no reason to believe features improtant for one will work in the same way for another. $\endgroup$– Matthew DruryCommented Jul 22, 2017 at 3:25
-
1$\begingroup$ @MatthewDrury I'll write this up as an answer, but if you'd prefer to make this comment into an answer, I'll delete my quotation. $\endgroup$– Sycorax ♦Commented Jul 3, 2018 at 15:21
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
From comments, Matthew Drury writes:
You shouldn't use xgboost as a feature selection algorithm for a different model. Different models use different features in different ways. Theres no reason to believe features important for one will work in the same way for another.
