I have seen a few useful examples on the SKlearn documentation page where in some situations, over-fitting can be handled to a reasonable extent by making sure that the splits leave each node with at least a certain number of samples/observations.
I'm assuming other software packages, like R, have this feature too, I don't really mind which implementation the answer comes from, because I'm asking more from the theoretical perspective. But for the sake of illustration, let's just use Python's syntax for a moment:
If we omit the
min_samples_leaf argument, it will default to 1, and that means the decision tree/random forest will only need 1 observation to justify a split -- which does seem somewhat prone to overfitting.
My Question is: How can we decide what the
min_samples_leaf argument for the general case should be? If we have a very large dataset, wouldn't 1 or even 5 be too small? Specifically, what formula might that entail; would we make it proportional to
n (total observations) or some other heuristic?