Very low out-of-bag score after applying Random Forest I am applying Random Forest to a matrix of 388 samples by 14 features.
The features are:


*

*nominal (5 categories) (1 feature)

*nominal (2 categories) (13 features)


The target variable is nominal (6 categories).
Computing the out-of-bag score I get a score of 0.4974, which means, if I understood well, that my classifier misclassifies half of the samples.
I am using 1000 trees, which are expanded until all leaves are composed by only 1 sample.
I am using the Random Forest implementation in Scikit-learn.
What am I doing wrong? I was thinking that maybe the number of samples is too large with respect to the number of features, but I am unsure on that.
 A: In worst case scenario, your features are completely unrelated to your target or related to your target in such a complicated way, that even RF cannot learn any reproducible pattern. A RF model will as default(no class weight, no stratification) assume same target distribution as of training set. If e.g. the most prevalent class make up 80% of the training set, the RF model can still use this information alone to roughly predict any new sample as member of this class, and achieve only a 20% class err.rate. In your case, if your training data is balanced such that each class is represented by 100%/6=16.7%, the expected worst performance is cross-validated 83.3% err.rate.
"What am I doing wrong?" - probably nothing, just too poor variables to predict your target any better. Try some 'feature engineering' or get some new variables. Maybe you realize 50% err.rate for your problem is not that bad at all. If you could predict with 50% err.rate the winner of the next 50 Tennis grand-slam tournaments, you probably could earn a fortune on sport betting.
"I was thinking that maybe the number of samples is too large with respect to the number of features" -That is never a problem. Feel free to discard a random fraction of your samples. It won't make your model better though.
I can warmly recommend the tutorial competition at kaggle: Titanic, which can teach you how to assess your RF model performance.
