# Explanation for the success of bagging

I'm reading Machine Learning - A First Course for Engineers and Scientists. On page 168 they give a rough explanation for why bagging works. I'm a little confused by their explanation.

They consider an ensemble of $$B$$ bootstrapped datasets $$\mathcal{T}^{(b)}$$ with $$b=1,2, \ldots, B$$. They denote the prediction from the $$b$$-th bootstrapped dataset $$\mathcal{T}^{(b)}$$ (taken from the original dataset $$\mathcal{T}$$) and evaluated at the point $$\mathbf{x}_\star$$ as $$\tilde{y}^{(b)}_\star \equiv y(x_\star; \mathcal{T}^{(b)})$$. They assume that the average of these predictions is $$\mathbb{E}[\tilde{y}^{(b)}_\star] = \mu^2$$ the variance is $$\mathrm{Var}[\tilde{y}^{(b)}_\star] = \sigma^2$$ and the average correlation between predictions is $$\text{avg cor}[\tilde{y}^{(b)}_\star] =\frac{1}{B(B-1)}\sum_{b\ne c}\mathbb{E}[(\tilde{y}^{(b)}_\star-\mu)(\tilde{y}^{(c)}_\star-\mu)] = \rho \sigma^2$$ They state that

"All base models, and hence their predictions, originate from the same data $$\mathcal{T}$$ (via the bootstrap), and $$\tilde{y}^{(b)}_\star$$ are therefore identically distributed but correlated."

Their explanation goes like this: the average of the bagged prediction is $$\mathbb{E}[\frac1B \sum^B_{b=1}\tilde{y}^{(b)}_\star] = \sigma^2$$ and the variance is $$\mathrm{Var}[\frac1B \sum^B_{b=1}\tilde{y}^{(b)}_\star] = \frac{1-\rho}{B} \sigma^2+\rho \sigma^2$$ If $$\rho<1$$, then variance is reduced by increasing bootstrap samples.

I'm a bit confused by exactly how they treat $$\tilde{y}^{(b)}_\star$$ as random variables i.e. what probability distribution are we averaging over in $$\mathbb{E}[\tilde{y}^{(b)}_\star]$$? Since the point is that the bootstrapping procedure decreases the variance in the bias-variance decomposition of the expected new data error, I'm supposing that the randomness is from draws of the original dataset $$\mathcal{T}$$. But are we supposing that the indices of bootstrap samples are fixed (i.e. $${\mathcal{T}'}^{(b)}$$ is always constructed from taking the datapoints with indices $$i \in \lbrace i^{(b)}_1, i^{(b)}_2, \ldots, i^{(b)}_n \rbrace$$ but $${\mathcal{T}'}^{(b)}$$ is random), or are they also random between draws of $$\mathcal{T}$$ ?

Also, can anyone give an intuitive explanation of what sort of data and model characteristics would increase $$\rho$$?

It's also random between draws, to ensure lower $$\rho$$.
In random forests, random subsets of variables (eg $$\sqrt{n_{features}}$$) are used as candidates to build trees to reduce $$\rho$$ (ie tree 1 is only allowed to use features 1,5,6, tree 2 is only allowed 4,7,8, etc)