Ok, I spent almost most of my day trying to figure this out.

I drew the picture below on my board(it can be seen better in zoom in) to have a better view on how a decision tree was taking its decision when working with continuous attributes from a homes' pricing dataset.

Now, I can't figure out if at the leaf/terminal node, is the tree suppose to give me the average price(target data) from all the rows within the specific branch path/conditions?

And so if in fact, this is the case, let say I am building a forest with ten trees, wouldn't my other trees do the same as the first tree.

I have been finding a lot of examples, but most of them are for classification rather than regression.

Thank you for the help!

enter image description here


1 Answer 1


The answer to your first question regarding the predictions at the terminal nodes is yes. You can think of your decision tree dividing up your data into disjoint subsets. For example, if you have 8 terminal nodes, each of your data will fall into one and only one of these 8 terminal node "groups".

From here, we can see it should be reasonable to choose the mean of the data points in a terminal node as the predicted value of that terminal node.

For your second question: if you are building multiple trees off of the exact same dataset using the same split criterion (e.g. variance reduction), you will get the same decision tree every time. If you're trying to build a random forest (https://en.wikipedia.org/wiki/Random_forest), you'll be generating new bootstrapped datasets to build additional trees, not re-using the same exact dataset for every tree.

  • 3
    $\begingroup$ +1. Note that there are two reasons why RFs grow different trees. One is that the learning data is bootstrapped, as you write. The other one is that the set of possible features on which to split is randomly chosen before each split. This helps "de-correlate" the trees and improves generalizability. $\endgroup$ Commented Oct 24, 2017 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.