I am trying to understand the ways that the predictions of each Tree in a Random Forest or each Fern in Random Ferns are combined to form a single response.
I did read somewhere (reference missing) that each tree/ferns result is counted as a vote for the class with the highest posterior probability in that leaf. Then all votes are counted and the highest vote is return.
This sort of democratic approach does not sound statistically sound to me. During training not all leafs are filled with the same amount of samples/information. Therefore I would consider some leafs response more powerful than others.
How can I account for the power of each leaf? What is the theory behind it?
EDIT: Concrete example
Assume we have three weak classifiers. For a single sample/datapoint each classifier returns a posterior distribution (learned from training set):
classifier A says P(class1) = 6/10, P(class2) = 4/10, --> class1
Classifier B says P(class1) = 6/10, P(class2) = 4/10, --> class1
Classifier C says P(class1) = 1/100, P(class2) = 99/100 --> class2
So if we would just count the democratic vote, the total classification would yield: class1.
Yet both A and B are less secure about their vote opposed to classifier C. How do i account for this classifiers certainty? Further, how to i account for the number of samples which ended up in the specific leaf, yielding each posterior. Image that Classifier A and B both got their posterior distribution each from 10 training samples, Classifier C in the other hand got his posterior from 100 samples. (Here the number of samples are those which end up in the specific leaf of the decision tree. Yet all classifiers have been trained with the same amount of training data - assume 200 samples)