Information gain in random trees When splitting attributes while constructing a random tree, I use information gain in order to determine the best value to split the tree on. I add nodes to the tree until a stopping criterion is met. What is the minimum value of information gain, to be used as a stopping criterion?
 A: The minimum information gain required for a split is a tunable parameter and probably should be determined using cross validation on a problem by problem basis. You can also run statistical significance tests for each split, addressing the question whether the split provides a statistically significant increase in information gain over a random split. Check out pages 12 and 13 from this PDF for an example of statistical tests on splits.
A: I agree with 

The minimum information gain required for a split is a tunable parameter

The most important factor to consider while tuning this parameter is bias-variance tradeoff:


*

*If the minimum information gain for stopping criteria is very low, the depth of the tree will be very high and consequently, the number of samples at each leaf node will be low. This is a high variance situation (over fitting).

*If the minimum information gain for stopping criteria is very high, the depth of the tree will be very low and consequently, the number of samples at each leaf node will be high. This is a high bias situation (under fitting).

