I am just looking to understand how mtry works in random forests. Please correct me if I am wrong.
When you specify mtry (say 10), it takes 10 random variables from your data set and examines them for one tree. So the next tree would take 10 more random variables, examine them, so on and so forth until it runs through the ntrees that you specify and then returns the average estimates for the best/most important variables?
No, that's not how this works.
Consider a single tree being added to a Random Forest (RF) model.
The standard recursive partitioning algorithm would start with all the data and do an exhaustive search over all variables and possible split points to find the one that best "explained" the entire data - reduced the node impurity the most. The data are split according to the best split point and the process repeated in the left and right leaves in turn, recursively, until some stopping rules are met. The key thing here is that each time the recursive partitioning algorithm looks for a split all the variables are included in the search.
Where RF models differ is that when forming each split in a tree, the algorithm randomly selects mtry variables from the set of predictors available. Hence when forming each split a different random set of variables is selected within which the best split point is chosen.
Hence for large trees, which is what RFs use, it is at least conceivable that all variables might be used at some point when searching for split points whilst growing the tree.
mtry variables selected at random when forming each split. The multiple trees of an RF is irrelevant to the question about mtry and how it factors into the algorithm.
$\endgroup$
Jun 10, 2014 at 17:37