Random forest variance I've been using RFs for about a month now (no experience before). What I've noticed so far that if I keep the configuration of a RF (number of trees, objective function, split conditions, etc) and rerun on the same data, I get very different results every time I do it: false positives and true positives vary from trial to trial quite a bit, which is annoying because I want a stable, predictable result. Is there any way to control it? Intuitively increase in tree number should take care of the problem, but it doesn't. I'd appreciate suggestions on this. 
 A: Two thoughts. 


*

*RF are often split until purity. This often means that there are many terminal nodes, each with a single observation. The final splits leading up to these nodes may not generalize very well because there are so few observations to work with at that depth of the tree. So you may get more generalizable trees with lower out-of-sample variance if you increase the minimum node size from 1 to something like 10 or more (depending on how much data you have -- this is another hyper-parameter, so you might profitably tune it). This also has the property of yielding consistent probability estimates, which can be desirable in some contexts.

*Increasing the number of trees will reduce the variance of the estimator. This is an obvious consequence of one of the CLTs -- each tree is a binomial trial, and the prediction of the forest is the average of many binomial trials. Moreover, the trees are iid in the sense that they are all fit on different re-samplings of the data and different random subsets of features. So you have iid binomial trails (which have finite variance because each trial is 0 or 1, i.e. has finite cardinality). This can make the predictions less volatile because the trees only have to explain chunks of your data, instead of each observation. So four times as many trials will cut the standard error of the mean in half.
There is extended discussion of some of these RF properties in Elements of Statistical Learning. The consistency property is discussed in Malley JD, Kruppa J, Dasgupta A, Malley KG, Ziegler A. Probability Machines: Consistent Probability Estimation Using Nonparametric Learning Machines. Methods of Information in Medicine. 2012;51(1):74-81. doi:10.3414/ME00-01-0052.
Finally, as a general observation, the best regularizer is more data, and better features usually beat a cleverer algorithm.
A: 
I mean more of a variance reduction, i.e. predictable result. I increased the number of trees, but it didn't help.

Due to the central limit theorem, and due to the fact Random Forests predictions are obtained through averaging, increasing the number of trees should help. The default in R is 500L, set this as high as you can support (I've often put it to 5000L, depending on the data).
The randomness in Random Forests come from both attribute bagging and bootstrap aggregating. You may also try to reduce the randomness either of those add.
Last, depending on how many features and how many samples you have, it might simply due to the data, and no amount of hyperparameter tinkering will solve it.

You mentioned in a comment:

I've tried quite a few, between 6 and 400. I use a regression-based
  feature selector that keeps all features with p-value below 0.05

And, as I said, I find feature selection mostly useless with Random Forests. The reasons are simple: you risk overfitting by doing that selection and Random Forests are good with large number of features. Let the forest decide which features are worthy, unless you have orders of magnitude more features than samples, then do some small reduction, just enough to remove noise features.
A: Adding to what users said, if you average many uncorrelated or little correlated variable you get a reduction in variance.
Define $X=\sum_{i}^{B}T_i$ where $T_i$ is a single decision tree, B number of decision tree.
$Var(X)=\frac{1}{B^2}\sum_i\sum_jcov(X_i,X_j)=\frac{1}{B^2}
\sum_{i} ( \sum_{i\neq j} Cov(X_i,X_j)+Var(X_i)) = 
\frac{1}{B^2}\sum_ i ( (B-1)\sigma^2\rho +\sigma^3)=\frac{B (B-1)\sigma^2\rho+B\sigma^2}{B^2}=\rho\sigma^2+\sigma^2\frac{1-\rho}{B}$.
First quantity decreases if $\rho$ decreases to zero ( uncorrelation) and second quantity decreases if B increases. 
