Suppose, you are working on a binary classification problem. And there are 3 models each with 70% accuracy. If you want to ensemble these models using majority voting. What will be the minimum accuracy you can get?
Majority voting guarantees you an accuracy of at least 70%. Ensemble model predictions are wrong if at least two base classifiers are wrong. So a maximum of 30% can be classified incorrectly.
- Acc = 70%: if two models came up with the same predictions.
- Acc > 70%: in any other case.
As a side note: Not so sure about calling this boosting. It's kind of boosting in the sense that it turns a bunch of weak learners into a strong learner… However, majority voting or even OneRule classifier as meta learners are (maybe) less misleading.