Scaling a dataset for Random Forest modelling is not necessary. However, if we have already done the scaling and normalization to the dataset, will it impact our Random Forest modelling?

  • 3
    $\begingroup$ It shouldn't matter. Also decision trees/RF should be invariant to scaling and normalization since they find partitions, which only depends on orderings. $\endgroup$
    – user327671
    Aug 8 at 17:52
  • $\begingroup$ Related: stats.stackexchange.com/questions/72231/… $\endgroup$
    – Adrian
    Aug 10 at 18:31

Any monotonic injective transformation of the features won't change the model wrt how it splits the data. The reason is the same as for why scaling is unnecessary: the random forest looks for partitions, and partitions only depend on how the data are sorted. If there is an optimal split on some scale, then by the definition of monotonic injective, the same split exists after transformation, and it's just as good (at splitting the training data, at least).

  • 2
    $\begingroup$ (+1) It's just the splitting point may change after the transformation, e.g. if we split the feature $x$ between $[1, 2]$, the split point will be $1.5$. But, if we split $x^2$ between $[1^2,2^2]$, then the split point will be $x^2=5/2\rightarrow x=\sqrt{5/2}$. This won't affect model build and training performance, but might have little effects on the test set. $\endgroup$
    – gunes
    Aug 8 at 19:18
  • $\begingroup$ It's fair to point out that there is a difference between training a model and measuring its performance on a holdout set, so I've edited the first sentence to clarify that the model training is unchanged. $\endgroup$
    – Sycorax
    Aug 8 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.