# Why are my random forest results so variable?

I'm trying to test random forest's ability to classify samples between 2 groups; There are 54 samples and varying numbers of variables used for classification.

I was wondering why the out-of-bag (OOB) estimates can vary as much as 5% from one another even when I'm using 50k trees? Is this something that bootstrapping could help with?

• You have to few samples. 50k trees doesn't make any sense with so few samples. The variation are most likely just one sample being incorrectly classified between runs.
– ThiS
Mar 22, 2017 at 14:54
• @ThiS I thought that increasing the number of trees would reduce the amount of variance I get. Is there a way to reduce it to effectively zero or know which one is the most accurate? Mar 22, 2017 at 15:06

Increasing the number of trees can reduce the variance of the estimate of something like $p(y=1|x)$, though. Consider the results from the central limit theorem: increasing the sample size can reduce variance of a statistic like an average, but not eliminate it. The random forest predictions are an average $\bar{x}$ of all the trees' predictions, and these predictions are themselves random variables (because of the bootstrapping and random subsetting of features; both happen independently, so votes are also iid). The CLT provides that $\bar{x}$ approaches a normal distribution $\bar{x}\sim\mathcal{N}(\mu,\frac{\sigma^2}{n})$, where $\mu$ is the true mean prediction and $\sigma^2$ is the variance of the trees' votes. (Votes take values of either 0 or 1, so an average of the votes has finite variance.) The point is that doubling the number of trees will cut the variance of $\bar{x}$ in half, but won't drive it to zero. (Except when $\sigma^2=0$, but we know that is not the case here.)