2
$\begingroup$

In a decision tree, when we search for an optimal split, we usually minimize root mean square deviation (RMSE). In addition to that we might forbid splits that give too small leaves (for example a leaf cannot contain less that 100 data points).

I wonder if there is a less "rigid" and more statistically driven approach. For example can we replace RMSE by another formula that contains leaves sizes (n1 and n2) explicitly?

For example, we might "dislike" splits with small leaves but still accept them, if the corresponding RMSE is very good. So, in other words, are there smooth analytical penalties on leaves sizes.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Conditional inference trees do something like this, they run some sort of test on each split and only split if certain criteria is met. I don't know about RMSE. $\endgroup$
    – user2974951
    May 17, 2022 at 12:45
  • $\begingroup$ See also stats.stackexchange.com/q/190014/232706 $\endgroup$
    – Ben Reiniger
    May 20, 2022 at 19:51
  • $\begingroup$ @user2974951 (+1) That can be a reasonable answer if you expand on it... $\endgroup$
    – usεr11852
    May 27, 2022 at 9:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.