Bagging is to use the same training for every predictor, but to train them on different random subsets of the training set. When sampling is performed with replacement, this method is called bagging (short for bootstrap aggregating). When sampling is performed without replacement, it is called pasting.
In other words, both approaches are similar. In both cases you are sampling the training data to build multiple instances of a classifier. In both cases a training item could be sampled and used to train multiple instances in the collection of classifiers that is produced.
In bagging, it is possible for a training sample to be sampled multiple times in the training for the same predictor. This type of bootstrap aggregation is a type of data enhancement, and it is used in other contexts as well in ML to artificially increase the size of the training set.
Computationally bagging and pasting are very attractive because in theory and in practice all of the classifiers can be trained in parallel. Thus if you have a large number of CPU cores, or even a distributed memory computing cluster, you can independently train the individual classifiers all in parallel.
using scikit-learn for performing bagging and/or pasting is relatively simple. As with the voting classifier, we specify which type of classifer we want to use. But since bagging/pasting train multiple classifiers all of this type, we only have to specify 1. The n_jobs parameter tells scikit-learn the number of cpu cores to use for training and predictions (-1 tells scikit-learn to use all available cores).
The following trains an ensemble of 500 decision tree classifiers (n_estimators), each trained on 100 training instances randomly sampled from the training set with replacement (bootstrap=True). If you want to use pasting we simply set bootstrap=False instead.
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
bagging_clf = BaggingClassifier(
max_samples=100, bootstrap=True, n_jobs=-1
y_pred = bag_clf.predict(X_test)