Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am just looking to understand how mtry works in random forests. Please correct me if I am wrong.

When you specify mtry (say 10), it takes 10 random variables from your data set and examines them for one tree. So the next tree would take 10 more random variables, examine them, so on and so forth until it runs through the ntrees that you specify and then returns the average estimates for the best/most important variables?

Thanks!

share|improve this question
    
In my understanding, the RF model does not inherently return the "best" variables; rather, each tree "votes" for class membership, according to the set of variables that were given to that particular tree. In most schemes, majority rules, but I suppose one could just as easily specify an alternative decision rule... –  user777 Jun 10 at 17:24
    
@user777 Most RF implementations do return variable importance measures, but this is not really relevant to how mtry affects the building of trees. In a regression RF, you don't get votes but contributions to MSE. –  Gavin Simpson Jun 10 at 17:40
    
@GavinSimpson Thank you for correcting my understanding. This exchange has been very helpful to me. +1 (: –  user777 Jun 10 at 17:56
add comment

1 Answer 1

up vote 5 down vote accepted

No, that's not how this works.

Consider a single tree being added to a Random Forest (RF) model.

The standard recursive partitioning algorithm would start with all the data and do an exhaustive search over all variables and possible split points to find the one that best "explained" the entire data - reduced the node impurity the most. The data are split according to the best split point and the process repeated in the left and right leaves in turn, recursively, until some stopping rules are met. The key thing here is that each time the recursive partitioning algorithm looks for a split all the variables are included in the search.

Where RF models differ is that when forming each split in a tree, the algorithm randomly selects mtry variables from the set of predictors available. Hence when forming each split a different random set of variables is selected within which the best split point is chosen.

Hence for large trees, which is what RFs use, it is at least conceivable that all variables might be used at some point when searching for split points whilst growing the tree.

share|improve this answer
    
Gavin, in your first paragraph, do you mean "decision tree" rather than "random forest" in the first sentence? (Apologies if I'm daft.) –  user777 Jun 10 at 17:21
    
No I meant consider a single tree in an RF model. I should separate it from the next sentence, but the point was, focus on a single tree; if using RPART, it would use all variables for all splits, but in RF the algorithm uses mtry variables selected at random when forming each split. The multiple trees of an RF is irrelevant to the question about mtry and how it factors into the algorithm. –  Gavin Simpson Jun 10 at 17:37
    
Thank you so much! Your explanation has helped me understand this correctly! –  user2615556 Jun 10 at 17:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.