# How does Weka combine the decision trees in a random forest?

When building the random forest, I am wondering if Weka combine the decision trees by averaging their probabilistic prediction or if Weka let each decision tree vote for a unique class?

Weka documentation does not tell much, but it refers to following paper

Leo Breiman (2001). Random Forests. Machine Learning. 45(1): 5-32.

and for more details you could check it. Slides from Data mining with Weka MOOC by one of Weka authors says that it

• Uses voting (or, for regression, averaging)
-- but weights models according to their performance

So basically: in classification case it uses voting weighted by performance and in regression case averages models weighting on their performance. However this does not seem to be well documented so if you want to be 100% sure you'd have to check the source code, or e-mail it's developers.

Wikipedia states the following: After training, predictions for unseen samples $x'$ can be made by averaging the predictions from all the individual regression trees on $x'$:

$$\hat{f} = \frac{1}{B} \sum_{b=1}^B \hat{f}_b (x')$$

or by taking the majority vote in the case of decision trees. In sum, there is an equation when regression trees are adopted. Voting is only accepted when the random forests employs decision trees.

• I don't think this is a helpful answer to OP's question. OP is asking about how Weka implements random forest, not what Wikipedia says. – Sycorax Feb 4 '16 at 14:42
• I agree with @user777. – Sean Easter Feb 4 '16 at 14:59