What is the default node splitting process carried by sci-kit's RandomForestRegressor when all features and target are continuous?

Question

I have some data containing several features, mainly continuous variables.

Implementing the randomForestRegressor algorithm from the sci-kit package in Python is relatively simple and results look OK. However, I need to know how the node splitting process is carried out by the randomForestRegressor algorithm.

I have been reading the documentation (https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) and (https://scikit-learn.org/stable/modules/ensemble.html#forest) but have not been able to find a direct answer.

My assumption is, that if the dataset is given as is or minmax normalized (which is still continuous form for all variables, only from 0 to 1 values) at each node, the splitting process will be done in a binary way over a randomly selected subset of features, at a value which minimizes target variable variance in the resulting split nodes? but there are other possibilities...

Does anybody know which is the exact process carried out by this package?

Answer 1

Scikit learn's implementation follows the original implementation of Breiman. At every split, a subset of features is selected (size depends on $mtry$ parameter). For each feature, the algorithm computes the drop in squared loss (eq. to variance) at every split point. The split (over all features) returning the best loss drop is then selected.

The drop in the loss is simply the squared loss of the current estimator minus the one that the new estimator would achieve by splitting at a given point.

The package should also be able to perform this step by only evaluating a subsample of all split points in case of large dataset, and also to evaluate the splits after binning the continuous variables in order to increase speed (like XGboost does).

If you need more specific details, you can read "Understanding Random Forests", which is the PhD thesis of one of the authors of the sklearn Random Forest package.