2
$\begingroup$

XGBoost will produce different values for feature importances with different hyperparameters on the same dataset. When using XGBoost as a feature selection algorithm for a different model, should I therefore optimize the hyperparameters first? Or there are no hard and fast rules, and in practice I should try say both the default and the optimized set of hyperparameters and see what really works?

$\endgroup$
2
  • 3
    $\begingroup$ You shouldnt use xgboost as a feature selection algorithm for a different model. Different models use different features in different ways. Theres no reason to believe features improtant for one will work in the same way for another. $\endgroup$ Jul 22, 2017 at 3:25
  • 1
    $\begingroup$ @MatthewDrury I'll write this up as an answer, but if you'd prefer to make this comment into an answer, I'll delete my quotation. $\endgroup$
    – Sycorax
    Jul 3, 2018 at 15:21

1 Answer 1

2
$\begingroup$

From comments, Matthew Drury writes:

You shouldn't use xgboost as a feature selection algorithm for a different model. Different models use different features in different ways. Theres no reason to believe features important for one will work in the same way for another.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.