# Bagging Ensemble Math

You are working on a binary classification problem with 3 input features and have chosen to apply a bagging algorithm (Algorithm X) on this data. You have set max_features = 2 and n_estimators = 3. Each estimator has an accuracy of 70%. Algorithm X aggregates the results of individual estimators based on maximum voting.

I get a solution. The maximum accuracy you can get is 79.85% (rounded to two decimal places). Here’s how to calculate it:

The probability of a single estimator being wrong is 30%. The probability of all three estimators being wrong is 0.3 * 0.3 * 0.3 = 0.027. Therefore, the probability of at least one estimator being correct is 1 - 0.027 = 0.973. The probability of all three estimators being correct is 0.7 * 0.7 * 0.7 = 0.343. Therefore, the probability of at least two estimators being correct is 3 * 0.7 * 0.7 * 0.3 = 0.441. The probability of all three estimators being correct or at least two estimators being correct is 0.343 + 0.441 = 0.784. Finally, the probability of the majority vote being correct is 0.784 + 0.5 * 0.027 = 0.7985. Therefore, the maximum accuracy you can get is 79.85% (rounded to two decimal places).

Is it correct? If yes, then can anyone explain 0.343 + 0.441 = 0.784.

• Why did you add $0.5\times 0.027$ at the end? That's a case where you know the majority vote is wrong because all 3 estimators are. This problem reads as "what is the probability of getting at least 2 heads out of 3 with P(head) = 0.7" (where the answer is $0.784$). Commented Jan 4 at 10:24
• Many thanks. In some blogs, I get the maximum accuracy of 100%. But they did not give any explanation. Is it possible to get 100% accuracy from here? 16 no question from this blog analyticsvidhya.com/blog/2017/09/… Commented Jan 4 at 11:22

Consider the following (10x3) table with 3 estimators (A, B, and C) where 1's represent right prediction for this row and 0's represent wrong prediction for this row. Each estimator got 7 out of 10 cases right. The final column is whether majority voting will predict the right final class (more than 50%).

A B C vote
1 0 1 1
1 0 1 1
1 0 1 1
1 1 0 1
1 1 0 1
1 1 0 1
1 1 1 1
0 1 1 1
0 1 1 1
0 1 1 1

As you can see it was able to achieve 100% accuracy.