1
$\begingroup$

Thanks to several online resources, including this Cross Validated question, I am clear that - within Random Forests - mtry represents the number of variables randomly sampled without replacement at each split of a tree.

What I am not clear on is if sampled without replacement means that a variable that was sampled for the first split, but passed on because another another randomly sampled variable provided greater outcome homogeneity, could be sampled again for the second split of the tree.

Here is a simple example:

  1. I have a binary classification problem.

  2. My dataset has 8 predictor variables: V1, V2, V3, V4, V5, V6, V7, V8

  3. I've set up my model where mtry = 3

  4. For the first split of the first tree, the three randomly selected variables to test are V2, V4, V7 and the algorithm has determined that V4 provides the maximum homogeneity of the three variables.

I understand that V4 is not an option for the randomly sampled variables of the second splits (2) because it is already being used for the first split.

My question is: Are V2 and V7 available to be randomly sampled again in both of the second splits along with the variables that were not sampled in the first split? i.e., are the variables available for sampling in both of the second splits V1, V2, V3, V5, V6, V7, V8?

$\endgroup$
2
$\begingroup$

All variables are available as potential candidates for every split. This includes all variables examined in the previous split and the variable used for the previous split. Of these potential candidates, $m$ will be examined, and the best chosen.

In this context, "sampled without replacement" only refers to the current split. It wouldn't make sense to sample with replacement because that would amount to duplicating features, which wouldn't add anything, and would reduce the number of features examined. Neither would it make sense to remove features considered in previous splits, because they may be good for splitting descendant nodes. Furtheremore, doing so could cause us to run out of variables to use.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ so in my example above, all variables (including V4) are available again for the second split? $\endgroup$ – bshelt141 Aug 10 '17 at 18:43
  • $\begingroup$ @bshelt141 Yes, that's right $\endgroup$ – user20160 Aug 10 '17 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.