What methods can be used to overcome the tie-breaking when using majority voting in ensemble? I read that Weighted majority voting can help; however, it wasn't effective on the dataset I am using in terms of the F1 score and it dropped significantly.
a) count the votes, select the class with more votes
b) count the votes weighted by the confidence (or probability) the base classifier assigns to its decision. Select the class with higher weighted vote.
c) count the votes. If there is a tie (and only if there is a tie) select the class with higher weighted votes for only the classes tied as best. (I think sklearn uses this at least in their OVO implementation)
None of them is "best". This third one is not equivalent to the 2nd and so it may be worth it to try.