# Is there a random forest variant which handles relationships between variables more elegantly?

Let's assume that I want to train a random forest classifier which predicts 1 if variables a/b = constant and 0 otherwise. If I have enough data, it should be possible to build decision trees which do just that.

However, I wonder if there is a random forest variant which automatically tries to combine two variables in different ways to see if there are more optimal split points.

Surely, if there are only few variables, I could just generate such feature combinations beforehand and use a standard random forest implementation. However, with lots of features, this quickly becomes a problem. The random forest implementation, on the other hand, could just pick random features and combine them on the fly.

Is anybody aware of a random forest implementation which does just that or research papers about similar ideas?

Random forests (RF) are essentially implementing an ensemble learning approach where the learner perform recursive partitioning of the training sample's feature space. Interactions between variables are captured as a variable $$x_b$$ being used to partition a subspace already partition by a variable $$x_a$$. One of RF's strength is exactly this, it automatically partition the training sample such that if such an interaction between $$x_a$$ and $$x_b$$ is relevant, it will be automatically picked up. We do not have to code this interaction.
(Side-note: A relation like the one initially mentioned $$C>\frac{x_a}{x_b}$$ can could be easily visualised through a two-variable partial dependency plot; it would materialise as a strong diagonal trend. To start with, methodologies like the one mentioned by Basu et al. would allow detecting the relevant $$x_a$$, $$x_b$$ interaction.)
• You could try that, yes. The Wright et al. 2016 paper mentions some relevant metrics, e.g. Pairwise permutation importance (PPI) and Joint importance by maximal subtrees (JIMS), that are available through R packages ( ranger and randomForestSRC respectively). (I think PPI requires a particular ranger version while JIMS is directly available through the find.interaction function.) – usεr11852 Dec 30 '18 at 21:31