2
$\begingroup$

I read this answer. Was still unable to understand how bagging reduces variance.

Is there any other way to explain it mathematically to a newbie ?

Edit

Can anybody explain me this excerpt from the other answer?

by averaging the outputs of $B$ trees, the variance of the final prediction is given by $p \sigma^2 + (1 - p)\sigma^2/ B,$ where $p$ is the pairwise correlation between trees.

$\endgroup$
4
  • 1
    $\begingroup$ Does this answer your question? How can we explain the fact that "Bagging reduces the variance while retaining the bias" mathematically? $\endgroup$
    – develarist
    Commented Sep 12, 2020 at 23:01
  • $\begingroup$ I have mentioned in my Q statement that I checked "that" answer already, but failed to understand. It would be very useful if it's described in some other way. $\endgroup$
    – Debbie
    Commented Sep 12, 2020 at 23:04
  • 2
    $\begingroup$ What part of the other answer do you not understand? Can you be more specific about what you understand from the other answer and what is not clear to you? $\endgroup$
    – Sycorax
    Commented Sep 12, 2020 at 23:19
  • $\begingroup$ Hi, I edited my question- mentioned what I couldn't understand in the other answer. Plz see once. $\endgroup$
    – Debbie
    Commented Sep 13, 2020 at 10:33

1 Answer 1

1
$\begingroup$

There is a formula for the variance of adding a linear combination of random variables here:

https://en.wikipedia.org/wiki/Variance#Weighted_sum_of_variables.

In your case the $a_i$ is $1/B$.

This gives the answer but in terms of variance and covariance. Thus you just change the covariance to correlation via $covariance(X_i, X_j) = correlation(X_i, X_j) sd(X_i) sd(Y_i)$.

$\endgroup$
1
  • $\begingroup$ thanks for reply. I'll get back soon after finishing my current assignment. $\endgroup$
    – Debbie
    Commented Sep 16, 2020 at 20:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.