OpenCV provides an implementation of random forest named random trees and derived from a decision tree class. One parameter to train the random forest is the maximum depth, which in the provided examples is typically between 10 and 20. I learned that random forest is generally grown to its full depth and no pruning is done and, therefor, other random forest implementations do not provide a parameter of maximum depth. Why then, without consulting the source code, which might give the answer, does the OpenCV implementation provide this parameter and is it meaningful to limit the maximum depth in a random forest?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
there are lots of implementations that do provide max depth parameter. It is basically used as a method of reducing complexity of the tree classifier and therefore the variance of the estimator.
-
$\begingroup$ thanks for the information. i have two follow up questions: is this relevant to regression or to classification as well? is there a rule or procedure how to choose the optimal depth? $\endgroup$ Commented Aug 1, 2013 at 10:15
-
$\begingroup$ There are no established rules to choose optimal depth. Generally, Random forests work well when grown to full depth. Especially in classification. By default, the vanilla implementation in R grows trees until there are 5 obs. in the final nodes for regression and 1 for classification. From Experience I have seen an improvement for regression by growing the trees to maximum depth for some data sets. For some classification problems, especially if the classes are very imbalanced; setting larger final node sizes sometimes helps. Really, you have to play around with your specific problem. $\endgroup$– JEquihuaCommented Aug 2, 2013 at 3:25
-
$\begingroup$ Thanks for the clarification. If I understood you correctly, maximum depth is one parameter to further optimize the classifier in certain cases, but other parameters bear greater weight, e.g. the number of trees. $\endgroup$ Commented Aug 2, 2013 at 5:01
-
$\begingroup$ Meanwhile I re-read the random forests chapter of Hastie's et al "The Elements of Statistical Learning", which confirms the above statements. $\endgroup$ Commented Aug 5, 2013 at 6:19
$\begingroup$
$\endgroup$
1
I've played with the max-depth parameter extensively and I think that they provide this in case your data set is large. In those instances, it may take a very long time for the trees to be fully grown. This is then compounded if you then ask for many trees (say, 1000+). Also, it is important to note that the Random Forest code actually uses the Decision Tree code for generating its trees and there's actually a hard coded maximum value for the max-depth parameter (which is set to 25). In other words, if you specify any value larger than 25 then it will just change your parameter to max-depth=25. It is possible to change the source code and then recompile (I've done this).
-
1$\begingroup$ I just figured this out the hard way after a lot of training. Wish I had stumbled upon this sooner. $\endgroup$ Commented Nov 29, 2018 at 22:11