0

I created a random forest. When observing the trees that compose it in many of them the first variable to make the split is "age".

But here my doubt arises. The values to make the split change. For example:

 Tree 1 : Does the split if age > 5 
 Tree 4 : Does the split if age > 6
 Tree 18: Does the split if age >15

My question is, why is this happening? If the variable "age" is going to make the first split, shouldn't it always have the same value to do it?

I´m ussing scikit-learn and RandomForestClassifier

1 Answer 1

1

Random forests resample the data for each of the trees that are built, using sampling with replacement from the full set of observations. Thus, with very high probability, every tree is based on a different set of observations. Given that, we would expect that, even if the same variable is chosen to make the split at the top of the tree, the split values themselves would change.

4
  • Thanks for your answer. This is an interpretability problem, isn't it? The question that will come after saying that "age" is an important variable is: "at what age?""starting from what age is my dependent variable most likely to be 1 and not 0? And in this case I will not know if it is > 5 , > 6 or >15 . Any idea how to deal with this?
    – ozo
    Commented Dec 26, 2018 at 2:22
  • There are several tools for aiding interpretability of random forests and gradient boosting machines; ceteris paribus (pbiecek.github.io/ceterisParibus/cheatsheets/…) and the dalex (r-bloggers.com/dalex-how-would-you-explain-this-prediction) packages in R being two that work very well.
    – jbowman
    Commented Dec 26, 2018 at 3:15
  • Do you know any in Python?
    – ozo
    Commented Dec 26, 2018 at 3:21
  • No, but there might be some. There might not be, though...
    – jbowman
    Commented Dec 26, 2018 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.