Not able to figure out the difference between hard and soft voting in context to ensemble based methods

In my view they are doing the same thing.

In hard voting, the voting_classifier counts the number of each class_instance and then assigns to a test_instance a class that was voted by majority of the classifiers. In soft computing, there is a probability term coming that takes the average of probabilities for each class and then uses it to classify the test_instance.

How are they different from one another? Any mathematical formulation etc. that could make the difference between them more obvious?

For soft voting classifier, I also read that "it gives more weight to highly confident votes". How is it implemented mathematically. Is it the weighted average that the author is talking about or anything else?


Suppose you have probabilities:

0.45 0.45 0.90

Then hard voting would give you a score of 1/3 (1 vote in favour and 2 against), so it would classify as a "negative".

Soft voting would give you the average of the probabilities, which is 0.6, and would be a "positive".

Soft voting takes into account how certain each voter is, rather than just a binary input from the voter.


Not the answer you're looking for? Browse other questions tagged or ask your own question.