Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm using scikit-learn to build a decision tree (or a random forrest) for a regression problem. I have continuous variables as my regressors. I wonder to know how the splits in a regression decision tree in scikit-learn are being calculated? That is, how it searches through the space of all possible split points to determine which one is the best split point for a continuous variable? The implementation is based on CART.
I've seen some using the
Ignoring all optimizations, what you should do to find the best split for a given continuous feature is to sort your samples (say we have $n$ of them) and try all $n-1$ split points to see if which one is the best. For example, when your samples ar sorted, e.g. sorted version is $x_1,x_2,...x_n$, you need to try thresholds between pairs $(x_1,x_2)$, $(x_2,x_3)$, ...$(x_{n-1},x_n)$, in which you have $n-1$ of them. When this is done for all candidate features, you find the best split. Of course, there will be several optimizations such as presorting, using the information available from ascendants etc, which scikit-learn uses in its Splitter implementation.
