From what I understand, bagging reduces variance of prediction for a model. Though OLS is on the "low variance" part of the spectrum, I wish to understand anyway the implications of resampling from the training set and getting n estimates of the regression coefficient via OLS. Then, the prediction looks like:
$y'=X/n(b_1+b_2+....b_n)$
The prediction without bootstrap is:
$y''=Xb_1$
So it looks like the variance of the prediction has reduced (b_i's are uncorrelated conditional on the sample) . The prediction is still unbiased though. Is this 'free lunch' where variance of prediction can be reduced without compromising bias?