I created a random forest. When observing the trees that compose it in many of them the first variable to make the split is "age".

But here my doubt arises. The values to make the split change. For example:

 Tree 1 : Does the split if age > 5 
 Tree 4 : Does the split if age > 6
 Tree 18: Does the split if age >15

My question is, why is this happening? If the variable "age" is going to make the first split, shouldn't it always have the same value to do it?

I´m ussing scikit-learn and RandomForestClassifier


Random forests resample the data for each of the trees that are built, using sampling with replacement from the full set of observations. Thus, with very high probability, every tree is based on a different set of observations. Given that, we would expect that, even if the same variable is chosen to make the split at the top of the tree, the split values themselves would change.

  • $\begingroup$ Thanks for your answer. This is an interpretability problem, isn't it? The question that will come after saying that "age" is an important variable is: "at what age?""starting from what age is my dependent variable most likely to be 1 and not 0? And in this case I will not know if it is > 5 , > 6 or >15 . Any idea how to deal with this? $\endgroup$ – ozo Dec 26 '18 at 2:22
  • $\begingroup$ There are several tools for aiding interpretability of random forests and gradient boosting machines; ceteris paribus (pbiecek.github.io/ceterisParibus/cheatsheets/…) and the dalex (r-bloggers.com/dalex-how-would-you-explain-this-prediction) packages in R being two that work very well. $\endgroup$ – jbowman Dec 26 '18 at 3:15
  • $\begingroup$ Do you know any in Python? $\endgroup$ – ozo Dec 26 '18 at 3:21
  • $\begingroup$ No, but there might be some. There might not be, though... $\endgroup$ – jbowman Dec 26 '18 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.