1
$\begingroup$

I would like to know how the results of different decision trees are aggregated (average) in random forest regression.

If I have a numeric target variable and 10 predictors, each decision tree is build with a randomly selected subset and randomly selected predictors.

Does every decision tree produce an output covering the entire range of possible target values?

Does each leaf node of one decion tree contain a unique target value?

How are the single leaf nodes averaged with the leaf nodes of the other decision trees?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

In random forests (w/o any modifications), resulting target values of each decision tree are purely averaged. And, each leaf node of one decision tree should contain a unique target value, since it is also the average of member samples in that leaf. But, you cannot guarantee the entire range of possible target values for a specific decision tree because of bootstrap sampling. Your highest and lowest possible target values may not be member of a decision tree you've constructed; or even if they're members, you might be averaging them with others corresponding to the same leaf node.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for your answer. How is the averaging done? Let's say we have 10 decision trees each with leaf nodes A to Z. Is the averaging done like this: (A1 + ... + A10)/10 ? $\endgroup$
    – yPennylane
    Sep 14, 2018 at 10:55
  • $\begingroup$ Yes, here check it out from Tree bagging subsection: wikiwand.com/en/Random_forest $\endgroup$
    – gunes
    Sep 14, 2018 at 10:57
  • $\begingroup$ I thought they were weighted averaged, depending on expected tree performance on things like OOB samples. $\endgroup$
    – EngrStudent
    Sep 21, 2018 at 16:04
  • 1
    $\begingroup$ @EngrStudent, no they are not weighted. At least not in random forests w/o any modifications. $\endgroup$
    – Marjolein Fokkema
    Feb 25, 2021 at 23:17
  • 1
    $\begingroup$ @MarjoleinFokkema - you are correct. There is a basic, and then there are many variations on it. We try to be always learning. $\endgroup$
    – EngrStudent
    Feb 26, 2021 at 1:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.