5
$\begingroup$

Suppose, you are working on a binary classification problem. And there are 3 models each with 70% accuracy. If you want to ensemble these models using majority voting, what is the minimum accuracy you can get?

$\endgroup$

2 Answers 2

0
$\begingroup$

Majority voting guarantees you an accuracy of at least 70%. Ensemble model predictions are wrong if at least two base classifiers are wrong. So a maximum of 30% can be classified incorrectly.

  • Acc = 70%: if two models came up with the same predictions.
  • Acc > 70%: in any other case.

As a side note: Not so sure about calling this boosting. It's kind of boosting in the sense that it turns a bunch of weak learners into a strong learner… However, majority voting or even OneRule classifier as meta learners are (maybe) less misleading.

$\endgroup$
1
  • $\begingroup$ What's the easiest way to prove this? Exactly 70% accuracy can be achieved in other situations, e.g. classifier A is correct on the first 70%, B on the last 70%, and C on a middle 70%. (C always concurs correctly with one of A,B.) $\endgroup$ Commented Jun 18, 2021 at 18:59
0
$\begingroup$

The minimum is actually 55%. Here was my first example achieving 60% accuracy, which helps inform the improvement in the next paragraph. Each row is one of the models, and each column is 10% of the data; an x indicates the model being incorrect on that decile.

xxAxAAAAAA
xBxxBBBBBB
CxxxCCCCCC

That fourth column of all xs is wasteful, so we're going to have the three models all be correct on a subset of the data of proportion $t$, and with the remaining $1-t$ they will cycle the rest of their $0.7-t$ correct answers such that only one of them is correct on any point. This means that $3\cdot(0.7-t)=1-t$, and so $t=0.55$.

This is, to me, clearly the worst possible, because we've "wasted" correct votes maximally. But I don't have an elegant proof, and this should definitely already exist somewhere...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.