1
$\begingroup$

I am currently writing a blog where I plan to do a regression model using Random Forest. Random Forest averages the estimated prediction from many decision trees that are fitted on to a bootstrap from the dataset while choosing K<=M variables randomly from all the M dependent variables.

However, I recently came across this presentation. One of the slides says something like this:

enter image description here

Here, the author is building a regression model using Random Forest with Residual Sum of Squares as the splitting criterion. During the splitting process, it is stated that a split is "too close to the edge."

My question is, what does an "edge" mean in this case? And why not choose a split if it is too close to the "edge"?

Thanks in advance.

$\endgroup$
1
$\begingroup$

This is a worked out example of the recursive splitting done by tree-based models.

After going through the linked slides, I think what Cutler tries to say is that if you were to split at $3.07$, you would be rather close to the minimum value of that variable in the sample. Hence, a split near the 'edge' would yield one node containing almost all of the (remaining) observations and the other containing only one or a few. To avoid this imbalance, he suggests splitting slightly further away from the extremes.

$\endgroup$
1
$\begingroup$

The author probably means that the cut leaves only a small number of data points in one of the leaf nodes. It's hard to tell what he / she is thinking out of context, but it probably has to do with the fact that a split that hardly changes the data at all is almost the same as making no split, so you might as well stop there (or use a different variable). Essentially, if the "best" split occurs at the boundary of a variable's range, that indicates that the response is fairly homogeneous with respect to that variable and it probably shouldn't be used for splitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.