Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Currently i am using RF toolbox on MATLAB for a binary classification Problem
Data Set: 50000 samples and more than 250 features
So what should be the number of trees and randomly selected feature on each split to grow the trees? can any other parameter greatly affect the results?
Pick a large number of trees, say 100. From what I have read on the Internet, pick $\sqrt{250}$ randomly selected features. However, in the original paper, Breiman used about the closest integer to $\frac{\log{M}}{\log{2}}$.
I would say cross-validation is usually the key to finding optimal parameters, but I do not know enough about random forests.
Number of trees the bigger, the better. You almost can't overshoot with this parameter, but of course the upper limit depends on the computational time you want to spend on RF.
The good idea is to make a long forest first and then see (I hope it is available in MATLAB implementation) when the OOB accuracy converges.
Number of tried attributes the default is square root of the whole number of attributes, yet usually the forest is not very sensitive about the value of this parameter -- in fact it is rarely optimized, especially because stochastic aspect of RF may introduce larger variations.
Number of trees the bigger, the better : agreed.
Number of tried attributes will depend. If you already have some a priori about the way the information is spreading or not among the features. If the information is shared by many features, better results would come up with smaller value of that parameter. While, on the other hand, if only a few features are carrying the information, you should use larger values. In other words, with many relevant variables : smaller values are better and with many irrelevant variables : bigger values are better.
