# What is the purpose of using duplicated data in resampling techniques (e.g., bagging/bootstrapping)?

With bootstrapping and bagging, we resample from the dataset and end up building a model or estimating some sample statistic using the sampled data, which typically consists of at least $$33\%$$ duplicate data.

My questions are:

(1) Why do even need to use the duplicated data? Why don't we simply discard them, and just use the unique data for each bag?

(2) When the duplicated data is used, aren't you putting more emphasis on those data during the learning process? If so, doesn't that introduce some bias into the model?

• The gist of bootstrap is that you're treating the discrete, empirical distribution as a population. When you sample from a discrete distribution, you might get duplicated values. – Dave Sep 3 '20 at 18:23
• There's a similar question and answer here. I like to think of a duplicated draw as representing another member of the underlying population that has values close to the case that's being duplicated from your original sample. – EdM Sep 3 '20 at 21:07
• @EdM Just read your answer in that post. So essentially, the idea is if we're allowed to sample with replacement, as with bootstrapping, then each draw from the pool is independent? But then this seems to come with the cost that the samples within the bootstrapped dataset is not entirely independent. If you resampled without replacement, then your bootstrapped dataset samples would be independent (assuming the original pool of data was independently generated)? – anonuser01 Sep 3 '20 at 23:15
• Having identical values doesn't make samples dependent. Think about rolling a die repeatedly: there are only 6 possible outcomes so you will get many values repeated as you roll over and over. Nevertheless, the distribution among those values can provide information about whether the die is fair or not. In bootstrapping, the data that you have are considered to be a discrete representation of the underlying population. Sampling without replacement then means that subsequent samples depend on what happened in prior samples. That's dependency in sampling even if all values are distinct. – EdM Sep 4 '20 at 14:54