# Random forest - binary classification vs. regression?

I have a dataset that I'm trying to classify into 2 groups, A and B, using a random forest model. I know the true grouping and I'm trying to see how well I can model it using the other available variables. I've tried 2 different approaches that I thought would be equivalent, but which are actually giving me quite different results:

1. Reading in the grouping as a (non-numeric) factor in R, growing a classification forest, and taking the proportion of trees that vote for group A as my prediction.
2. Constructing an indicator variable for membership of group A, growing a regression forest, and taking the ensemble prediction as usual.

The split between the 2 groups is roughly 90-10 A vs. B. I'm growing 240 trees from ~200k observations of the same variables. I've left most of the settings at the defaults for the R randomForest package, but to keep the processing time down to a manageable level I've increased the node size to 200. The results are as follows:

1. In the vast majority of cases, all 240 trees vote for A. The average predicted chance of any one observation being in A is about 99.9%. Worse still, not a single member of group B gets a majority of votes for group B!
2. I get a wide range of predictions, with the mean prediction lying close to the observed mean of ~90%.

How can two apparently similar methods give such different results?

As for how I ended up trying this - I was initially trying to classify my dataset into a larger number of groups, of which B was one, but I noticed that B was being classified almost 100% incorrectly. The other groups are all much better behaved, even though most of them make up a far smaller proportion of my data.

• how do you construct the indicator variable for membership of group A? Why would you think these approaches are smiliar? On the other hand, you have a highly unbalanced problem (90% A and 10% B if I understood correctly) since the default setting of random forests is simply to take the majority vote that's why A always wins. There are much more A observations. You can try balancing the model by doing randomForest(..., sampsize=c(size,size),...) where size has to be small than the size of your smaller class. Maybe like 60% of it since the objective of random forest is to introduce variation. May 29 '13 at 3:35
• I'd have expected on average about 90% of the trees in the classification forest to vote for A, but what I'm actually seeing is very close to 100% every time. I'm interested in the % of trees voting for each group, not just which one has a majority of votes. May 29 '13 at 7:26
• I think it could either be beacause you are using the default setting so it's taking 0.632 of A and 0.632 of B for each tree it builds. But each tree is not grown till there is only one observation in each node so the proportion is less favorable than a tree built on the whole data set. It could also be because you have only 240 trees, random forests are like nearest neighbour mechanisms, so if you dont have enough trees the random sampling might not grab your nearest. Another thing could simply be that your clases are not separable. If you do a balanced forest is the classification good? May 29 '13 at 16:41
• How many training examples do you have? It is important. Oct 28 '13 at 18:17

Due to the class imbalance, you should have a look at the probabilities that your forests outputs (I'm not familiar with the random forest R package, but I think there is an option (type="prob") in the predict function that will give you a matrix of class probabilities.