# Can RandomForest multiplicatively combine features?

I have a relatively good understanding of how RandomForest mechanically works. However, here's what I want to understand: can RF model a multiplicative relationship?

For example, if I have features A and B predicting target variable C. Let's say the true relationship is A x B=C . Would RF be able to model this? Or would I have to have (A x B) as an input feature?

Thank you!

• Random forests can't even model additive relationships in the sense of reproducing them accurately within nonempty intervals. In what sense, then, do you mean "model" in this context?
– whuber
Commented May 2, 2022 at 19:39
• @whuber Fair point. I guess I just mean, from a practical/reasonable perspective, can I expect an RandomForest to pick up on that kind of relationship without me explicitly including the AxB feature as a column in the dataset. Commented May 2, 2022 at 20:29
• In that respect, you have gotten a good answer!
– whuber
Commented May 2, 2022 at 20:31

A tree can form a lot of splits of the form $$A>a_i$$ or $$B>b_i$$. Enough of such splits in a single tree can give you a step function that can then approximate just about any function of $$A$$ and $$B$$. In RF you usually limit how deep each tree gets, but instead average over lots of trees, which in combination again can give you a step function in $$A$$ and $$B$$ that can then approximate just about any function of $$A$$ and $$B$$.
However, if your trees are, say, 2 deep, that just gives you a single step in your step function per tree, so it takes a lot of trees, before you approximate a linear relationship such as $$A\times B$$ really well. If you give the RF the feature $$A\times B$$ instead, it needs half the depth for the same approximation. I.e. the functional relationship becomes easier for the model to fit.
Nevertheless, if you don't have all that much training data, that will limit how smooth the output of the RF can be no matter whether it gets $$A$$ and $$B$$, or additionally $$A \times B$$ as features.