Classification with randomForest in R - always predicts 0

Question

I have a data frame of approximately 11500 records.

Each row of the frame has a response variable, isfastes, and a number of other predictors (size, cores, timeout, queue, and selection) and two other variables, name and 'loggedtime'.

The data was generated by running a command on a provided input (identified by name) with various arguments (the predictors). I am trying to find out which combination of arguments typically leads to the fastest execution time (isfastest) regardless of the input. When I run a random forest as

randomForest(isfastest~cores+queue+timeout+selection+size, data=data)

the results are:

      0 1 class.error
0 11392 0           0
1   118 0           1

Essentially, the randomForest method is always predicting "no". At the very least I'd expect cores alone to be significant, as when I look at the set of isfastest == 1 rows, there are far more records where cores == 64 than otherwise (as I would expect).

What could be causing randomForest to consistently return 0?

Answer 1

Your classes are very unbalanced as 99% of your data are in class 0. By just always classifying to class 0, your algorithm is right 99% of the time and it doing an excellent job in terms of classification accuracy! However, this isn't practically useful.

There are two ways to handle this in Random Forests.

1. Bootstrap your data while over-sampling the rare class and under-sampling the common class.

2. Use a "weighted" random forest where miss-classification of the minority class is assigned a higher cost. You can do this with the classwt option in the randomForest function.

Answer 2

Adding onto TrynnaDoStat's answer, sometimes, the prior probabilities using the classwt option tends to affect the algorithm more drastically than desired. Another thing you can do is predict the probabilities rather than the class, and use a different threshold for assignment of the results. For instance, this would be code if you decided to use 0.3 as the threshold (instead of 0.5) for assigning a 1:

rf1 <- randomForest(isfastest~cores+queue+timeout+selection+size, data=data)
predictions <- predict(rf1, data=data, type='prob')
predictedClass <- ifelse(predictions > 0.3, 1, 0)

To determine the threshold you desire (depending on how much you are willing to increase your false positive rate in order to increase your true positive rate), receiver operator curves are a great way to view the data. The ROCR package does this very nicely:

library(ROCR)
ROCRprediction <- prediction(predictedClass, data$isFastest)
ROCRcurve <- performance(ROCRprediction, "tpr", "fpr")
plot(ROCRcurve, colorize=TRUE)