I've started reading about Random Forests and one of the attributes that appeals to me is that they are good at dealing with independent variables that interact with one another. Does Random Forest also "automatically" transform variables into, for example, the square of the variable? Is there a good reason to square or perform other transformations of a variable (of course dependent on the situation)? For example if I have y = x1 + x2 + x3 is there anything to be gained by running the model y = x1 + x1^2 + x2 + x2^2 + x3 + x3^2?


1 Answer 1


Random Forests are Decision Tree - based. So all they do is comparisons of your individual variables with some thresholds. Squaring your parameters simply will shift these thresholds, introducing no change to the actual output. So, in theory, RF's are not sensitive to rescaling.

  • 2
    $\begingroup$ A little more care in this analysis would be welcome, because squaring is not a one-to-one transformation. Thus a comparison of the square to a threshold is not quite the same thing as a comparison of the original variable to a threshold. This issue can be fixed up, provided the transformation is sufficiently nice, but what about the case of multivariate transformations of variables, such as $x_1x_2$? That's no mere "rescaling." $\endgroup$
    – whuber
    Dec 15, 2015 at 22:44
  • 2
    $\begingroup$ If your variable allows negative values squaring is not a monotonic transformation, and the forest will therefore pick that. $\endgroup$
    – Firebug
    Jan 12, 2017 at 2:25
  • $\begingroup$ I don't agree with this answer. Decision trees are built based on groups variances and if your transformation is not linear, then, in the other scale, the "most heterogeneous" grouping can change. As an example, a feature is uniformly distributed on (0,0.1) for a group and on (0.95,1.05) for the other, now take the non-linear transformation x^200 and there is a high chance those guys between 0.95 and 1.00 will be classified with the first group . $\endgroup$
    – Freguglia
    Sep 8, 2018 at 21:58
  • $\begingroup$ @Freguglia, that's not a problem here, because the "groups variances" refers to the dependent variable, not the features. All a decision tree cares about is the relative ordering of features, so (assuming $x$ is positive), $x^{200}$ carries the same information as $x$ for a tree model. $\endgroup$ Jan 10, 2021 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.