Combine mutliple predictions This question had been asked several times in here, but I think I have something new to add.
I'm interested in predicting if some specific event will happen (binary classification). I have two distinct methods that generate a prediction about it. The first methods, an XGBoost model, gives a binary response and the probability of the event (as the probability of being in that class). The second method is an empirical model that has as output a binary classification, but with I can estimate the probability of event given the classification.
Now, the question is, how to combine the two?
My proposal, and here is where I think I didn´t saw anything similar, is the following:
if both classifiers agree on prediction, then that's the prediciton.
When they disagree, I want to compute the weighted probability of event, where I will give more weights to extreme probabilities. The logic is that there is more information in a classifier that tells you there is a 90% or a 1% change of event compared to one that says 50%.
To do that I compute the weight $w_i$ as $|\ln\frac{p_i}{1-p_i}|$, with $p_i$ as the probability of event. What do you think
 A: 
there is more information in a classifier that tells you there is a 90% or a 1% change of event compared to one that says 50%

Why do you think so?
If one classifier outputs more extreme probabilities, that first of all indicates that it is more confident. This certainly does not indicate that it is better. I can easily build you an extremely confident classifier that is extremely bad! (Simply output "probabilites" of 0% or 100% at random.)
The goal of any probabilistic classifier should be good calibration: when it outputs a 90% chance that a case belongs to class A, it should in fact be A in 90% of cases. If it is A in 85% or in 95% of cases, then the classifier is just as poorly calibrated as if it output 50% probability but was correct only in 45% or 55% of cases. Take a look at proper scoring-rules to assess calibration.
That said, of course your particular problem and classifier could be improved by such a scheme. That is an empirical question. I would just be careful about claiming that higher confidence implies higher competence. That simply doesn't work. Neither for classifiers nor for humans.
