Can a decision tree automatically detect the effect on the dependent variable from the product/quotient of two independent variables?

Question

For example, when I use the xgboost algorithm, there are two continuous variables X1 and X2, do I need to specify the product X1*X2 explicitly at the beginning? Or the algorithm can automatically pick up the effect of the X1*X2?

Answer 1

Yes and no. By having a sufficiently deep tree (at least two splits deep) and splitting on both $x_1$ and $x_2$, tree based model like xgboost (or LightGBM or catboost) can eventually approximate (given enough data) any relationship between $x_1\times x_2$ and your prediction target of interest. Of course, if you know that some function $f(x_1\times x_2)$ is a predictor of your outcome of interest, then specifying a feature that is that exact transformation of $x_1$ and $x_2$ is what you should do.

What to do in practice? If you have a lot of data and are not that sure that the interaction matters, perhaps don't specify it as a feature. If you don't have a lot of data and suspect it's an important feature, you probably want to put this feature into xgboost. If you are inbetween - try it out using an appropriate cross-validation scheme.

Answer 2

The answer to your question depends on what class of split rules you allow in the fitting of a decision tree. If the only class of allowable splits are on a single variable you will never be able to capture the interaction behavior described in the post. In fact what you will see that allows you to diagnose something like this has occurred is a repetition of splits in the decision tree on the two interacting variables and the repeated splits may occur several times. If you allow classes of splitting rules that allow for polynomials up to the order of the interaction you think may occur in your data (here 2nd order) then you will be able to capture the behavior in the decision tree that is fit to the data.