# Combine probabilistic forecasts with weighting

Suppose I want to compute the probability that Argentinia wins the worldcup semifinal Argentinia - Croatia. I have two independent sources of information about this probability, whereby source A says that Argentinia will win with probability $$p$$ and source B says that Argentinia will win with probability $$q$$. The two sources are truly independent, i.e. they themselves make their assessement on independent information (say A based on previous match statistics, B based on current player fitness levels).

In addition, I have variable trust in my sources. Say, when $$a=0$$ -> I don't trust A at all, $$a=1$$ -> I trust A completely. And $$b=0$$ -> I dont't trust B at all, $$b=1$$ -> I trust B completely. In other words, I weigh my sources.

My question: What is the optimal formula for the combined probability $$P$$ of these two sources?

Two my mind, the correct formula $$P(a, b, p, q)$$ should fulfill the following properties:

1. If I trust neither source ($$a=b=0$$):$$\quad P=0.5\quad$$ (i.e., I have to guess)
2. If I trust both sources completely ($$a=b=1$$):$$\quad P=(p+q)/2$$
3. If I trust source A completely ($$a=1$$), but B not at all ($$b=0$$):$$\quad P=p$$
4. If I trust source B completely ($$b=1$$), but A not at all ($$a=0$$):$$\quad P=q$$

Edit: I have found a formula that at least fulfills the above criteria, but rather based on intuitive guessing than principled reasoning:

$$P=a(1-\frac{b}{2})p+(1-\frac{a}{2})bq+\frac{1}{2}(1-a)(1-b)$$

The formula seems to make sense intuitively. For instance, if b=1, i.e. I trust source B completely, source A can at most receive "half of the weight", because of the factor $$(1-\frac{b}{2}=1-\frac{1}{2}=0.5)$$ in front of A's probability $$p$$, and this maximum weight of 0.5 for A is only reached when if I have also full trust in A ($$a=1$$). The term $$\frac{1}{2}(1-a)(1-b)$$ provides sort of the intercept of the 0.5 guessing probability when one trusts neither source, i.e. $$a=b=0$$.

Edit #2: My question was underspecified. An additional criterion of linearity is necessary, i.e.:

1. The probability $$P$$ should be linear in the weighting factors $$a$$ and $$b$$.

With this fifth criterion it is relatively easy to derive the formula in my first Edit (see my answer). Thanks to @whuber for pushing me towards this.

• This has no objective solution because the meaning of the "trust" quantity is not defined. If you could provide some kind of clear, operational characterization of "trust," then possibly this question is answerable.
– whuber
Dec 12, 2022 at 17:39
• Yes, that is a good point. My idea was that the trust parameters $a$/$b$ linearly scale between the two extremes given as critera 1. and 2. above. If $a=b=1$, my predicted probability is based entirely on $p$ and $q$, i.e. as the average $(p+q)/2$. If $a=b=0$, I discard both sources and I end up with guessing probability 0.5. If, for instance, $a$ is 0, I discard A completely, and if $b$ is also small, e.g. 0.1, it I would only have a slight preference away from 0.5 towards the probability given by source B. Dec 12, 2022 at 17:57
• I am not sure though whether it is sufficiently operationalized by saying that $a$/$b$ scale linearly between these extremes. Unfortunately I don't have a clearer concept at the moment. However, my formula in the edit seems to be in line with my idea. Dec 12, 2022 at 17:58
• @whuber: in my edit to the post I now provide an intuitive explanation of the formula I derived by intuitive guessing. Maybe this helps clarfying how the trust variables $a$ and $b$ are thought to operate. Dec 12, 2022 at 18:48
• do you have an actual problem in mind? If so please provide the actual problem rather than trying to abstract it yourself. Note that even though the sources of information are not directly related, doesn't mean the data is independent. eg fitness levels and match statistics are likely correlated. Dec 14, 2022 at 9:39

As rightly pointed out by @whuber, the question was underspecified. However, I believe that with the additional information provided in the comments that the sources are weighted linearly, it has an objective solution. I found this solution first by intituitive guessing, but it turns out that its more formal derivation is straightforward.

One idea is to say that there is a third "source", the "guessing source" with probability 0.5, which only comes into play when I can neither fully trust source A (i.e. $$a<1$$) nor source B ($$b<1$$). What I need then is a formula that weighs the probabilities $$p$$, $$q$$ and $$\frac{1}{2}$$ in accordance with the conditions stated in the question. I can first specify the most general linearly weighted form as follows:

$$P=(\alpha_1 a+\alpha_2b+\alpha_3 ab+\alpha_4)p + (\beta_1 a+\beta_2b+\beta_3 ab+\beta_4)q + (\gamma_1 a+\gamma_2b+\gamma_3 ab+\gamma_4)\frac{1}{2}$$

Using condition (1), i.e. if $$a=b=0 \;\rightarrow\;P=0.5$$, it is immediately clear that $$\alpha_4=0$$, $$\beta_4=0$$ and $$\gamma_4=1$$. Thus $$P$$ simplifies as follows:

$$P=(\alpha_1 a+\alpha_2b+\alpha_3 ab)p + (\beta_1 a+\beta_2b+\beta_3 ab)q + \frac{1}{2}(\gamma_1 a+\gamma_2b+\gamma_3 ab+1)$$

Using condition (3), i.e. if $$a=1$$, $$b=0 \;\rightarrow\;P=p$$

it is $$\alpha_1 p+\beta_1 q+\frac{1}{2}(\gamma_1+1)=p$$

From this I can conclude that $$\alpha_1=1$$, $$\beta_1=0$$ and $$\gamma_1=-1$$

Using condition (4) and by symmetry of $$a$$ and $$b$$, it must also be that $$\alpha_2=0$$, $$\beta_2=1$$ and $$\gamma_2=-1$$.

Thus, my probability simplifies as follows:

\begin{align} P & = (a+\alpha_3 ab)p + (b+\beta_3 ab)q + \frac{1}{2}(\gamma_3 ab-a-b+1)\\ & = a(1+\alpha_3 b)p + (1+\beta_3 a)bq + \frac{1}{2}(\gamma_3 ab-a-b+1) \end{align}

Now, conditions (2) to (4) imply more generally that if I trust one source completely (i.e. $$a=1$$ or $$b=1$$), I discard the "guesser". For instance, if $$a=1$$ and $$b=x$$ with $$0\le x\le 1$$, I know that the guessing term must be zero:

$$\frac{1}{2}(\gamma_3 ab-a-b+1)\stackrel{a=1, b=x}{=}\frac{1}{2}(\gamma_3 x-1-x+1)=0\;\rightarrow\;\gamma_3=1$$

Thus:

\begin{align} P & = a(1+\alpha_3 b)p + (1+\beta_3 a)bq + \frac{1}{2}(ab-a-b+1) \\ & =a(1+\alpha_3 b)p + (1+\beta_3 a)bq + \frac{1}{2}(1-a)(1-b) \end{align}

For the final paramaters $$\alpha_3$$ and $$\beta_3$$, I can make use of the rationale that if I trust one source fully, the other source can contribute at most half of the weight. For instance, if $$a=1$$, I know that the factor $$1+\beta_3 a$$ in front of B's probability $$q$$ should be $$\frac{1}{2}$$ irrespective of the value of $$b$$:

$$1+\beta_3 a \stackrel{a=1}{=} 1+\beta_3 = \frac{1}{2}\;\rightarrow\;\beta_3=-\frac{1}{2}$$

By symmetry between $$a$$ and $$b$$ it must also be that $$\alpha_3=-\frac{1}{2}$$.

Thus, the final probability is:

$$P=a(1-\frac{b}{2})p + (1-\frac{a}{2})bq + \frac{1}{2}(1-a)(1-b)$$

I'm sure this is not the most elegant proof, but for me personally it's only important to know that the formula can be proven.

• No proof of this sort can be convincing, because any solution can be altered by applying any increasing bijective function $f:[0,1]\to[0,1]$ for which $f(1/2)=1/2$ to all trust values, thereby preserving all four of your criteria. The value for $P$ would likely change, but no other aspect of the question would be any different.
– whuber
Dec 14, 2022 at 15:10
• @whuber Thanks for your comment, you probably have a point, but I'm out of depth here. I wonder: since I ended up with a unique solution anyway, in my derivation I must have made an additional assumption not covered by my question nor by my additional assumption of linearity (the latter of which is manifested in my starting formula for $P$, which contains only terms linear in $a$/$b$). Can you pinpoint this assumption? Or put differently: which additional criterion/criteria would I need to specify in order for my solution to be the unique solution? Dec 14, 2022 at 16:27
• Your assumption is linearity in $a$ and $b.$ In less mathematical terms, I am pointing out that you could relabel all the numbers in the interval $[0,1],$ keeping $0,1/2,$ and $1$ fixed, without changing anything about the problem, but such a relabeling generally gives a different solution. This is the same as saying your numbers $a$ and $b$ have only a relative meaning but no inherent quantifiable meaning.
– whuber
Dec 14, 2022 at 17:39
• Thanks! This is what I acknowledged in the beginning of my answer though, i.e. that the original question was missing the additional criterion of linearity. Dec 15, 2022 at 7:12
• If you want to make additional assumptions, then please write them into the question.
– whuber
Dec 15, 2022 at 14:41