For example, when I use the xgboost algorithm, there are two continuous variables X1 and X2, do I need to specify the product X1*X2 explicitly at the beginning? Or the algorithm can automatically pick up the effect of the X1*X2?

  • 1
    $\begingroup$ Welcome to the site! Trees can consider high-order interactions automatically. $\endgroup$ – Randel Jan 23 '16 at 21:11
  • $\begingroup$ Just make sure your max_depth is at least 2 to find 2-way interactions. $\endgroup$ – Dex Groves Sep 2 '16 at 22:35

The answer to your question depends on what class of split rules you allow in the fitting of a decision tree. If the only class of allowable splits are on a single variable you will never be able to capture the interaction behavior described in the post. In fact what you will see that allows you to diagnose something like this has occurred is a repetition of splits in the decision tree on the two interacting variables and the repeated splits may occur several times. If you allow classes of splitting rules that allow for polynomials up to the order of the interaction you think may occur in your data (here 2nd order) then you will be able to capture the behavior in the decision tree that is fit to the data.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.