1
$\begingroup$

I'm using scikit-learn to build a decision tree (or a random forrest) for a regression problem. I have continuous variables as my regressors. I wonder to know how the splits in a regression decision tree in scikit-learn are being calculated? That is, how it searches through the space of all possible split points to determine which one is the best split point for a continuous variable? The implementation is based on CART.

I've seen some using the

$\endgroup$

1 Answer 1

1
$\begingroup$

Ignoring all optimizations, what you should do to find the best split for a given continuous feature is to sort your samples (say we have $n$ of them) and try all $n-1$ split points to see if which one is the best. For example, when your samples ar sorted, e.g. sorted version is $x_1,x_2,...x_n$, you need to try thresholds between pairs $(x_1,x_2)$, $(x_2,x_3)$, ...$(x_{n-1},x_n)$, in which you have $n-1$ of them. When this is done for all candidate features, you find the best split. Of course, there will be several optimizations such as presorting, using the information available from ascendants etc, which scikit-learn uses in its Splitter implementation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.