# How splits are calculated in Decision tree regression in python

I'm using scikit-learn to build a decision tree (or a random forrest) for a regression problem. I have continuous variables as my regressors. I wonder to know how the splits in a regression decision tree in scikit-learn are being calculated? That is, how it searches through the space of all possible split points to determine which one is the best split point for a continuous variable? The implementation is based on CART.

I've seen some using the

## 1 Answer

Ignoring all optimizations, what you should do to find the best split for a given continuous feature is to sort your samples (say we have $$n$$ of them) and try all $$n-1$$ split points to see if which one is the best. For example, when your samples ar sorted, e.g. sorted version is $$x_1,x_2,...x_n$$, you need to try thresholds between pairs $$(x_1,x_2)$$, $$(x_2,x_3)$$, ...$$(x_{n-1},x_n)$$, in which you have $$n-1$$ of them. When this is done for all candidate features, you find the best split. Of course, there will be several optimizations such as presorting, using the information available from ascendants etc, which scikit-learn uses in its Splitter implementation.