Random Forest for Regression: How does a decision tree decides the value of a terminal node when outcome is many continues values?

Question

Ok, I spent almost most of my day trying to figure this out.

I drew the picture below on my board(it can be seen better in zoom in) to have a better view on how a decision tree was taking its decision when working with continuous attributes from a homes' pricing dataset.

Now, I can't figure out if at the leaf/terminal node, is the tree suppose to give me the average price(target data) from all the rows within the specific branch path/conditions?

And so if in fact, this is the case, let say I am building a forest with ten trees, wouldn't my other trees do the same as the first tree.

I have been finding a lot of examples, but most of them are for classification rather than regression.

Thank you for the help!

Answer 1

The answer to your first question regarding the predictions at the terminal nodes is yes. You can think of your decision tree dividing up your data into disjoint subsets. For example, if you have 8 terminal nodes, each of your data will fall into one and only one of these 8 terminal node "groups".

From here, we can see it should be reasonable to choose the mean of the data points in a terminal node as the predicted value of that terminal node.

For your second question: if you are building multiple trees off of the exact same dataset using the same split criterion (e.g. variance reduction), you will get the same decision tree every time. If you're trying to build a random forest (https://en.wikipedia.org/wiki/Random_forest), you'll be generating new bootstrapped datasets to build additional trees, not re-using the same exact dataset for every tree.