0
$\begingroup$

What's the difference, if any at all, between max_depth and max_leaf_nodes in sklearn's RandomForestClassifier for a simple binary classification problem?

If the model always grows trees in a symetric fashion, one would assume setting max_depth = 5 is equivalent to setting max_leaf_nodes = 32.

The fact that sklearn gives us 2 options suggests that might not be the case.

$\endgroup$
2
  • 1
    $\begingroup$ A tree of 32 nodes can have a depth far greater than 5. $\endgroup$
    – whuber
    Sep 9 at 13:43
  • $\begingroup$ That's what I suspected. Is there any documentation on RF grows trees leaf-wise when max_leaf_nodes is passed? $\endgroup$
    – Jingwei Yu
    Sep 9 at 13:45
1
$\begingroup$

As @whuber points out in a comment, a 32-leaf tree may have depth larger than 5 (up to 32). To answer your followup question, yes, when max_leaf_nodes is set, sklearn builds the tree in a best-first fashion rather than a depth-first fashion.

From the docs (emphasis added):

max_leaf_nodes : int, default=None

Grow trees with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.

and in the source code:

        # snipped from much earlier, line 231 in the permalink above:
        max_leaf_nodes = -1 if self.max_leaf_nodes is None else self.max_leaf_nodes
        ...
        # Use BestFirst if max_leaf_nodes given; use DepthFirst otherwise
        if max_leaf_nodes < 0:
            builder = DepthFirstTreeBuilder(
                 ...
            )
        else:
            builder = BestFirstTreeBuilder(
                ...
            )
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.