I have some ambiguity about dividing training dataset in Bagging tree.

In fact I have found in this article About Decision Tree Ensembles- Bagging

That : the idea is to create several subsets of data from training sample chosen randomly with replacement.

I want to understand which of the next solutions is bagging technique: if the total dataset is D (Without the target value y)

Solution 1:

  1. Splitting D to X_train and X_test :

  2. Splitting X_train to sevral subtraining sets like : X_train1, X_train2, X_train3, X_train4 etc. where X_train1 + X_train2 +X_train3+X_train4 +....=X_train.

  3. Finally training each dataset separatly and generating different model for each trained dataset.

  4. Average of all the predictions (generated trees) is used to get the unique final output tree


Solution 2:

  1. Splitting D to X_train1 and X_test1 :
  2. Training the dataset and generating the first decision tree
  3. Splitting D to X_train2 and X_test2
  4. Training the dataset and generating the second decision tree
  5. etc.
  6. Average of all the predictions (generated trees) is used to get the unique final output tree
  • $\begingroup$ Solution 2 looks like k-fold cross validation $\endgroup$
    – Elenchus
    Aug 25, 2020 at 13:28

1 Answer 1


Solution one is correct.

You set apart the test set (that will work as an indepentent dataset), and train individual trees on randomly sampled subsets of the training data with replacement.

In the end, you test your model in the initially separated test set, and average the predictions and the result is the random forest prediction.

It wouldn't be much profitable to have an individual test set for each tree, since each individual tree was trained in a limited sample!

  • $\begingroup$ "each tree only has available a random subset of features " it seems you are talking about Random Forest but here I am talking about the initial solution of Bagged tree before improving with random Forest $\endgroup$
    – baddy
    Aug 26, 2020 at 7:17
  • $\begingroup$ @baddy thanks for the correction! Edited the answer :) Still, solution 1 would be the correct one $\endgroup$
    – Johanna
    Aug 26, 2020 at 8:59
  • $\begingroup$ Thanks for your time $\endgroup$
    – baddy
    Aug 26, 2020 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.