Combine mutliple predictions

This question had been asked several times in here, but I think I have something new to add.

I'm interested in predicting if some specific event will happen (binary classification). I have two distinct methods that generate a prediction about it. The first methods, an XGBoost model, gives a binary response and the probability of the event (as the probability of being in that class). The second method is an empirical model that has as output a binary classification, but with I can estimate the probability of event given the classification.

Now, the question is, how to combine the two?

My proposal, and here is where I think I didn´t saw anything similar, is the following:

if both classifiers agree on prediction, then that's the prediciton. When they disagree, I want to compute the weighted probability of event, where I will give more weights to extreme probabilities. The logic is that there is more information in a classifier that tells you there is a 90% or a 1% change of event compared to one that says 50%.

To do that I compute the weight $$w_i$$ as $$|\ln\frac{p_i}{1-p_i}|$$, with $$p_i$$ as the probability of event. What do you think

• I think thats an interesting question, especially because the prediction itself is binary. If i understand your approach right, it would mean, that each time model1 gets a weight of > 1 it takes the "win" and if it gets a weight < 1 model 2 takes the win. If im right the breakingpoint towards model 1 should be between probability 0.73 and 0.74. Is this what you want to archive? – TinglTanglBob Mar 13 '19 at 10:23