why increasing diversity in ensemble method cause lower variance?

Question

I am reading hands on machine learning .

Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher bias than pasting, but this also means that predictors end up being less correlated so the ensemble’s variance is reduced.

I think in this situation model do not have many unique instances and it cause for raising bias because it can not predict best parammetr but I dont have any Idea why variance decreases.
I know about bias-variance trade off but I specially want to know what is happening in this situation.

Answer 1

Let $X_1,...,X_n$ be identically distributed random variables with variance $\sigma^2$.

If they are independent:

$$Var(\frac{1}{n}\sum\limits_{i=1}^nX_i) = \frac{1}{n}\sigma^2$$

If they are correlated with positive pairwise correlation $\rho > 0$, we have:

$$\frac{Cov(X_i,X_j)}{\sigma^2}=\rho \iff Cov(X_i,X_j) = \rho\sigma^2$$

$$\begin{aligned} Var(\frac{1}{n}\sum\limits_{i=1}^nX_i) &= \frac{1}{n^2} \times \left[\sum\limits_{i=1}^nVar(X_i) + 2\sum\limits_{1\leq i<j\leq n}^nCov(X_i,X_j)\right] \\ &= \frac{\sigma^2}{n}+\frac{n-1}{n}\rho\sigma^2 \end{aligned}$$

As you can see, as $\rho$ decreases (less correlation) variance decreases, and vice versa. The variance reduction effect is largest when individual trees are uncorrelated ($\rho=0$).