Does the optimal number of trees in a random forest depend on the number of predictors?

Question

Can someone explain why we need a large number of trees in random forest when the number of predictors is large? How can we determine the optimal number of trees?

Answer 1

Random forest uses bagging (picking a sample of observations rather than all of them) and random subspace method (picking a sample of features rather than all of them, in other words - attribute bagging) to grow a tree. If the number of observations is large, but the number of trees is too small, then some observations will be predicted only once or even not at all. If the number of predictors is large but the number of trees is too small, then some features can (theoretically) be missed in all subspaces used. Both cases results in the decrease of random forest predictive power. But the last is a rather extreme case, since the selection of subspace is performed at each node.

During classification the subspace dimensionality is $\sqrt{p}$ (rather small, $p$ is the total number of predictors) by default, but a tree contains many nodes. During regression the subspace dimensionality is $p/3$ (large enough) by default, though a tree contains fewer nodes. So the optimal number of trees in a random forest depends on the number of predictors only in extreme cases.

The official page of the algorithm states that random forest does not overfit, and you can use as much trees as you want. But Mark R. Segal (April 14 2004. "Machine Learning Benchmarks and Random Forest Regression." Center for Bioinformatics & Molecular Biostatistics) has found that it overfits for some noisy datasets. So to obtain optimal number you can try training random forest at a grid of ntree parameter (simple, but more CPU-consuming) or build one random forest with many trees with keep.inbag, calculate out-of-bag (OOB) error rates for first $n$ trees (where $n$ changes from $1$ to ntree) and plot OOB error rate vs. number of trees (more complex, but less CPU-consuming).

Answer 2

The number of trees in the Random Forest depends on the number of rows in the data set. I was doing an experiment when tuning the number of trees on 72 classification tasks from OpenML-CC18 benchmark. I got such dependency between optimal number of trees and number of rows in the data:

Answer 3

Accordingly to this article

They suggest that a random forest should have a number of trees between 64 - 128 trees. With that, you should have a good balance between ROC AUC and processing time.

Answer 4

i want add somthings if you have more than 1000 features you and 1000 rows you can't just take rondom number of tree .

my suggest you should first detect the number of cpu and ram before trying to launch cross validation in find the ratio between them and number of tree

if you use sikit learn in python you have option n_jobs=-1 to use all process but the cost each core requeire copy of data after that you can tris this formula

ntree = sqrt (number of row * number of columns)/numberofcpu