Are Random Forests trained with the whole dataset?

Question

I was reading "Hands On Machine Learning" by Aurelien Geron, and the following text appeared:

As we have discussed, a Random Forest is an ensemble of Decision Trees, generally trained via the bagging method (or sometimes pasting), typically with max_samples set to the size of the training set.

The max_samples argument defines how many elements of the data set go into each model trained, in this case, to the Decision Tree Classifiers. My question is, how does this make sense? Because in theory, if we passed the whole training set to each individual model, wouldn't all the models be exactly the same? Wouldn't it make more sense to train each model with a random part of the data set, in order to get completely different classifiers each time? So it can accurately do predictions.

This is how you would create a Random Forest:

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

bagging_classifier = BaggingClassifier(
  base_estimator = DecisionTreeClassifier(), # Model to use
  n_estimators=500, # Number of models to train
  max_samples=100, # Amount of samples to train each model (Putting the length of the whole dataset is what he's proposing)
)

Answer 1

All previous answers are correct.

I would like just to clarify one point of your question:

My question is, how does this make sense? Because in theory, if we passed the whole training set to each individual model, wouldn't all the models be exactly the same?

As you have seen in previous answers, we are doing Bootstrapping, so the weak decision trees we are creating are going to be different since they have different dataset.

But, imagine you set the bootstrap=False. Will you obtain the same week trees since you have the same dataset? The answer is NO.

Why? Because Random Forest selects also randomly some features to create every decision tree. So, you will have same dataset, but different features you are playing with.

The hyperparameter to tune this idea is called max_features.

Answer 2

As noticed in the comment, random forest uses bootstrap resamples of the training data. What this means is that for each tree we sample randomly with replacement the max_samples number of observations from the training data. When using bootstrap in statistics you generally want the number of bootstrap observations to be equal to the number of observations in your data, because you want the bootstrap samples to resemble the original data. Using max_samples higher than the number of observations in the training data would be rather pointless, but sometimes people may choose it to be smaller to speed up the computations. This would be a bad idea in statistics because using smaller bootstrap samples would not give you an accurate estimate of things like standard errors, but when training random forest you are only concerned with making predictions, not inference. In such a case, you need to empirically verify what are the consequences for the quality of the predictions if you make the max_samples number smaller.

Answer 3

From the explanation of the BaggingClassifier:

max_samples: The number of samples to draw from X to train each base estimator (with replacement by default, see bootstrap for more details).

So, since the samples are drawn with replacement, it's a bootstrap sample with the same size of the whole dataset, but not exactly the same dataset.