Random Forests overfitting/unbalanced classes? Suppose I am using random forests where the classes are highly unbalanced.
How do you detect over fitting and what can you do to avoid it? Breiman says in his paper that random forests do not overfit, but others say that they can? If overfitting does exist(i.e. correlated trees) what is the best course of action to counteract that, and why does Breiman say that Random Forests are impervious to overfitting?
Also, how do you deal with the fact that the class you want to predict is percent-wise so small(suppose you have 99% 1's and 1% 0's)? What are some key metrics to measure the overall model fit and how do you go about testing and training in such cases?
 A: In highly unbalanced datasets, how do you detect over fitting?
Use metrics that are robust against unbalanced datasets, like Precision, Recall or F1-score. In your example with 99% 1's and 1% 0's. A classifier that always predicts positive samples will an Accuracy of 0.99, but Precision, Recall or F1-score of 0.00
What can you do to avoid overfitting in unbalanced datasets?


*

*Cluster you positive samples into several clusters of the same size of the negative samples, i.e. move from binary classification to multi-label classification

*Sub-sample the positive samples in order to have a 50/50 dataset.

*Use other algorithm that deals naturally with unbalanced datasets, like anomaly detection methods.


Do Random Forest overfit?
Yes. Any classifier with high complexity respect to the training data will overfit. However, the overfitting will not increase when the number of single Decision Trees is increased.
How to avoid overfitting with Random Forest?


*

*Decrease the complexity of the Decision Tree: pre- or post-pruning

*Randomly drop features and/or samples per node

A: I personally think Random Forests are not impervious to overfitting. Overfitting is always a possibility, for any model. One possible way to counteract overfitting is by always using cross-validation.
If you want to detect overfitting, you can plot learning curves. Here, you are going to train the model multiple times, each time with a larger training set. Afterwards, you calculate the score on both the training set and the test set and plot these scores. Or, as Scikit Learn puts it: "A learning curve shows the validation and training error of an estimator for varying numbers of training samples." If the training error is low and the validation error is much higher, your model is overfitting.
To address your second question; when your data consists of 99% 1's and 1% 0's, this will certainly affect your final result!
