1
$\begingroup$

I am running Spark MLLib's Decisioin Tree model. While parameter tuning, I came across the minInstancesPerNode but I am not sure of the implications of setting it too low or too high. Can anyone help me understand its purpose and importance?

As per my understanding, it would be overfitting if there are very few instances in a node, since the tree would be too granular. I wanted to get more clarity.

$\endgroup$
2
$\begingroup$

The idea is that estimations made on nodes with very few samples are (in general) unreliable. It is a way to prevent overfitting.

With decision trees you attempt to minimize error by splitting at each node with the next feature yielding the lowest error rate, until done with all features. At each leaf, you assign the output label according to majority vote, i.e. the most frequent label on that node.

Now, suppose that for a given leaf, you have two possible outcomes, and the actual proportion are 60-40, and assume that you only have two or three samples for that node. Notice that when sampling from such a binomial distribution it is very likely to have all samples just one of the two cases, or the majority from the "wrong" one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.