I am using randomForest (the R package) to train a multi-factor, binary- classification model. In trying to dissect performance, I started feeding in individual factors to see how the RF treated them.
The first one was a continuous variable, with a range of -1.0 : 1.0. The factor is such that for 55% of the data (both training and OOS), values above 0.0 corresponds to class 1 and below 0.0 to class 2.
When I run this single-factor model through the randomForest however, it fails to learn this.
Here's an example:
f1 f1_signal rf_pred y_class 0.6 1 -1 1 -1.0 -1 -1 -1 0.6 1 -1 1 -0.9 -1 -1 1 -0.9 -1 1 -1 0.9 1 -1 1 -0.7 -1 -1 -1 0.7 1 1 -1 0.9 1 1 -1 0.6 1 1 -1 1.0 1 -1 -1
Just separating the prediction at +/-f1 (f1_signal) gives an accuracy of approximately 54%.
Putting this through the randomForest framework:
rf_pred <- randomForest(as.factor(y) ~ f1, data)
gives an accuracy of approximately 27% - substantially worse.
I understand that subsequent nodes may cause it to 'overlearn' the data, and have tried to address this by limiting maxnodes=2, and nodesize to 0.5*(training_sample_size), but still nothing.
My questions are:
1) Is there something I should be doing differently to allow the randomForest to learn this separation, or is randomForest simply ill-suited as a tool in a single-factor model?
2) Given its failure in learning a simple boundary, how do I view its learning in the larger, multi-factor space? Even running repeated OOS tests, can there be confidence (i.e. empirical measures) that actual 'learning' has happened? Is there a way to extract (for example), high-level factor splits, and apply them manually? I get that this last point is not really relevant in the RF world, but something along those lines.....
3) Is there another tool that might be better suited to building a model that does multi-factor, single-level splitting?