20
$\begingroup$

I do not understand exactly what is meant by node size. I know what a decision node is, but not what node size is.

$\endgroup$
24
$\begingroup$

A decision tree works by recursive partition of the training set. Every node $t$ of a decision tree is associated with a set of $n_t$ data points from the training set:

n_t is the size of each node

You might find the parameter nodesize in some random forests packages, e.g. R: This is the minimum node size, in the example above the minimum node size is 10. This parameter implicitly sets the depth of your trees.

nodesize from R random forest package

Minimum size of terminal nodes. Setting this number larger causes smaller trees to be grown (and thus take less time). Note that the default values are different for classification (1) and regression (5).

In other packages you directly find the parameter depth, e.g. WEKA:

-depth from WEKA random forest package

The maximum depth of the trees, 0 for unlimited. (default 0)

$\endgroup$
  • 1
    $\begingroup$ What are 'records'? Do you mean data points? Why is each node associated with a set of records? I understand random forests quite well, but I do not know what the jargon means. $\endgroup$ – wolfsatthedoor Jun 25 '15 at 7:08
  • $\begingroup$ Yes, I meant data point. Usually you might refer to data points as records, instances, or examples. $\endgroup$ – Simone Jun 25 '15 at 8:13
  • $\begingroup$ So is there a rule of thumb minimum node size to avoid overfitting trees? I'd imagine it depends on the size of the training data so perhaps a certain proportion of the dataset size? $\endgroup$ – Seanosapien Sep 20 '17 at 9:37
  • 1
    $\begingroup$ In random forests, trees are fully grown: node size is 1. Overfitting is avoided growing many trees. In decision tree it is more tricky. Trees are not fully grown and you have to perform pruning to avoid overfitting. $\endgroup$ – Simone Sep 20 '17 at 9:57
  • 1
    $\begingroup$ It looks like winnowing is some sort of feature selection to simplify the tree and avoid overfitting. I guess pruning a single tree is always beneficial. Instead, winnowing can sometimes decrease the accuracy but it simplifies the tree. $\endgroup$ – Simone Oct 2 '17 at 14:50
2
$\begingroup$

It is not clear if the nodesize is on the "in-bag" sampling or on the "out-of-bag" error. If it is on the "out-of-bag" sampling, it is slightly more restrictive.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.