How does an individual tree in a random forest misclassify ? I'm trying to understand how ensemble voting works for random forest. I have two sources of confusion. Let's say I have a single decision tree from a random forest built on a subset of the data. 


*

*Does it misclassify because, for instance, it was built on features
that have no real association with the thing we're trying to
classify? 

*Where does this misclassification come from? Is it from the out of bag sample that is withheld from the subset (i.e. the 36.8%) used to construct that particular tree? 
Thank you
 A: For the training set data:
1) If you grow the tree fully, without any pruning on early stopping on the training data, it will always give 100% performance on the sampled data used for training. The tree keeps dividing the n-dimensional space into k partitions, where k is less than the m samples used to train the tree. Each partition contains only one class of defects and any test defect in that partition is classified to that class.
2) Individual trees can misclassify on the out of bag sample. That is why Out Of Box Sample is used to estimate the OOB error, which can be used as an indicator of the test error.
3) Individual trees also misclassify because we try to get a diversity among the trees. One of approaches is to randomly select the features to make the partition. This leads to different partitions every time, which may not be representative of the real boundaries between the classes.
For the testing set data:
1) Similar to OOB error, we can have misclassifications on the test data, because there is not guarantee that the partition created by the trees represents the true boundaries separating the two classes, which is basically the concept of over fitting.
Overall in RF we hope to make the individual trees more diverse, so even though they make more mistakes individually, they make mistakes on different inputs, and the average of the votes gives us better performance. If all the individual trees were very good, but lacked diversity , they would make mistakes on the same set of inputs, and taking the average will still give us the wrong output.
