0
$\begingroup$

I'm using a random forest in R (randomForest) to predict a binary output (1,0) for a dataset that is heavily unbalanced. In this example let's assume the population has 1% 1's and 99% 0's.

Building the random forest on such unbalanced data is difficult and I get much better results when building it on a 50:50 sample. When predicting a validation set, I obtain the % of trees that predicted that data point to be a 1. For example, customer A has a 75% probability of being a 1 (based on the # of trees that predicted 1)

If I want to re-scale these predictions back to the original population ratio of 1:99, is there a good way to do this?

In the past I've used logistic regression, and I can adjust the intercept accordingly to down-scale the predicted probability.

Is there a good way to think about this from the RF point of view? Can I simply just down-weight the predictions from the 50:50 sample by 50 (50% down to 1%)?

Thanks in advance for any thoughts and help

$\endgroup$
3
$\begingroup$

Do not build a predictive instrument on a subset of the data. That is inefficient and arbitrary. Fix the fundamental problem that led you do to this: you chose a discontinuous improper accuracy scoring rule that is highly influenced by the prevalence of $Y=1$. Use methods that make random forests estimate probabilities in order to do this. Improper accuracy scores are optimized by bogus models, and create "classification" "rules" that do not transport to populations with different outcome prevalence.

$\endgroup$
0
$\begingroup$

One way to deal with a very skewed dataset is to rebalance it via bootstrap. It works like this. Let's say your rare class has label 1, the popular class has label 0. You bootstrap from you original data, whenever you get 1, keep that entry save it in a resample set, and up a counter by one. When it's a 0, check the counter, if you have more 0s than 1s, then discard this sample. Repeat until you have enough data for your model. This way, you'd get a balanced set.

BTW, this works for any model, not just random forest. This is known as oversampling/undersampling.

Weiss, G. M., McCarthy, K., & Zabar, B. (2007). Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs?. DMIN, 7, 35-41.

Farquad, M. A. H., & Bose, I. (2012). Preprocessing unbalanced data using support vector machine. Decision Support Systems, 53(1), 226-233

Just found this ref. Turns out there is a paper on exactly this topic:

Chen, C., Liaw, A., & Breiman, L. (2004). Using random forest to learn imbalanced data. University of California, Berkeley.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.