Some ambiguities about used training dataset in bagged trees

Question

I have some ambiguity about dividing training dataset in Bagging tree.

In fact I have found in this article About Decision Tree Ensembles- Bagging

That : the idea is to create several subsets of data from training sample chosen randomly with replacement.

I want to understand which of the next solutions is bagging technique: if the total dataset is D (Without the target value y)

Solution 1:

1. Splitting D to X_train and X_test :

2. Splitting X_train to sevral subtraining sets like : X_train1, X_train2, X_train3, X_train4 etc. where X_train1 + X_train2 +X_train3+X_train4 +....=X_train.

3. Finally training each dataset separatly and generating different model for each trained dataset.

4. Average of all the predictions (generated trees) is used to get the unique final output tree

OR

Solution 2:

1. Splitting D to X_train1 and X_test1 :
2. Training the dataset and generating the first decision tree
3. Splitting D to X_train2 and X_test2
4. Training the dataset and generating the second decision tree
5. etc.
6. Average of all the predictions (generated trees) is used to get the unique final output tree

Answer 1

Solution one is correct.

You set apart the test set (that will work as an indepentent dataset), and train individual trees on randomly sampled subsets of the training data with replacement.

In the end, you test your model in the initially separated test set, and average the predictions and the result is the random forest prediction.

It wouldn't be much profitable to have an individual test set for each tree, since each individual tree was trained in a limited sample!