I'm running some ensembles of several algorithms and I'm trying to establish possible rules for the final classification process.
Right now if have the class probabilities from 3 models. Its a binary classification so, in each prediction i have two columns
And each row has the probability for both classes.
Suppose Class_2 is the 'positive' outcome. I have a cutoff value where if the probability of Class_2 is greater than said value, I classify the row as Class_2, otherwise its Class_1. That is what I do with each model.
Now, since I have three models (I plan to extend it to 5 in the future), I've been trying the following rules, to build a 'blended' prediction.
- Average probability in each model, and check against the cutoff value
- Voting, return the cutoff value if a majority of models have a probability greater or equal to it
- Maximum probability of Class_2 per row, basically i return the greatest of the three probabilities of Class_2 and compare it againts the cutoff value.
Now, I'm not really sure if the third rule is right. Is it? On the other hand I'd like to know what other rules can I implement?