1
$\begingroup$

I would like to know how the results of different decision trees are aggregated (average) in random forest regression.

If I have a numeric target variable and 10 predictors, each decision tree is build with a randomly selected subset and randomly selected predictors.

Does every decision tree produce an output covering the entire range of possible target values?

Does each leaf node of one decion tree contain a unique target value?

How are the single leaf nodes averaged with the leaf nodes of the other decision trees?

Improve this question
$\endgroup$
1
$\begingroup$

In random forests (w/o any modifications), resulting target values of each decision tree are purely averaged. And, each leaf node of one decision tree should contain a unique target value, since it is also the average of member samples in that leaf. But, you cannot guarantee the entire range of possible target values for a specific decision tree because of bootstrap sampling. Your highest and lowest possible target values may not be member of a decision tree you've constructed; or even if they're members, you might be averaging them with others corresponding to the same leaf node.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for your answer. How is the averaging done? Let's say we have 10 decision trees each with leaf nodes A to Z. Is the averaging done like this: (A1 + ... + A10)/10 ? $\endgroup$ – yPennylane Sep 14 '18 at 10:55
  • $\begingroup$ Yes, here check it out from Tree bagging subsection: wikiwand.com/en/Random_forest $\endgroup$ – gunes Sep 14 '18 at 10:57
  • $\begingroup$ I thought they were weighted averaged, depending on expected tree performance on things like OOB samples. $\endgroup$ – EngrStudent Sep 21 '18 at 16:04
  • 1
    $\begingroup$ @EngrStudent, no they are not weighted. At least not in random forests w/o any modifications. $\endgroup$ – Marjolein Fokkema Feb 25 at 23:17
  • 1
    $\begingroup$ @MarjoleinFokkema - you are correct. There is a basic, and then there are many variations on it. We try to be always learning. $\endgroup$ – EngrStudent Feb 26 at 1:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.