I'm trying to tune the hyperparameters of my
RandomForestRegressor created in python with sklearn with
bootstrap = True using
GridSearchCV. The hyperparameters that I am trying to tune are
min_samples_split (for the sole reason that those are the hyperparameters given by an experiment I'm trying to duplicate). Both these parameters concern the number of samples needed to either split or form a leaf.
Let's say I have 1000 samples and I use 750 for the training and 250 for the testing data (
n_train = 750 and
n_test = 250). Now I perform 3-fold cross-validation on my training data. There should be 500 samples available for training and 250 for validation in every iteration of the cross-validation. Does every tree in this case get 500 randomly bootstrapped samples or 750? If it is 500 I don't see the point in tuning
min_samples_split, because the number of samples in every tree in the grid search is different from the number of samples in a tree when training on the complete training data (There every tree should be build from 750 randomly selected samples with doubles, triples, etc. allowed). Or does
GridSearchCV take this into account using the
RandomForestRegressor by doing the grid search with the number of samples that would be used by a full training?
TLDR: Is a tree in a random forest regressor using
GridSearchCV build from a number of samples equal to the size of the entire training data (
n_train) or from a number of samples equal to the size of the subset of the training data used for training (in this case
n_train/k * (k-1) = with
k = 2 gives