I have multiple weather forecasters who each use their own unique, independent calculation for prediction of the weather for the next day. We are only concerned with rain predictions to know if we may need an umbrella (i.e. we don't care about temperature or other factors).

Forecaster A, who has a track record of 34% accuracy of predictions, predicts it will rain tomorrow.

Forecaster B, who has a track record of 75% accuracy of predictions, also predicts that it will rain tomorrow.

Forecaster A and B agree on their forecasts 42% of the time (although this may not always be known).

Empirically (using rainfall average data) it would otherwise be 27% likely for rain tomorrow.

How would I practically calculate with Bayes Theorem a revised likelihood that it will rain tomorrow and what would the answer be? How would it change if we introduce n additional forecasters? I should add that I can work out a single-forecaster scenario as a straight-forward Bayes Theorem, but I've had trouble working out how to include/chain multiple forecasters of varying accuracy.

Many thanks in advance!

  • 1
    $\begingroup$ For a specific answer, we additionally need to know how correlated the predictions of A and B (and C) are. Roughly speaking, the less correlated they are, the more one can gain from combining their predictions. $\endgroup$ – Carsten Aug 14 at 14:33
  • $\begingroup$ Strongly related: stats.stackexchange.com/q/155817/35989 $\endgroup$ – Tim Aug 14 at 14:47
  • $\begingroup$ Thanks, @Tim, I found there was a lot of speculation in the answers so had difficulty determining the "right" answer. I posed my question to try to isolate a clear answer. $\endgroup$ – Miles Gillham Aug 14 at 22:01
  • $\begingroup$ Thanks, @Carsten, I'll update the question with an arbitrary value. What I'm trying to work out is an applicable formula and it's helpful to be aware of what additional data is needed to make it meaningful. $\endgroup$ – Miles Gillham Aug 14 at 22:05

As I already said in my comment, you can find answer to your question in the Combining probabilities/information from different sources thread. There is no single "right" answer, because there are multiple ways that you can employ to combine different forecasts, and in different cases different methods may work better, then the others. There is no single answer, because, as noticed by @Carsten in the comment, the forecasts are usually correlated and the amount of the correlation is unknown.

Same is true about using Bayes theorem in here. Bayes theorem is

$$ P(A\mid B) = \frac{P(B \mid A) P(A)}{P(B)} $$

so to use it, you would need to know the conditional probability

$$ P(B \mid A) = \frac{P(A, B)}{P(A)} $$

This is something you usually don't know (if you do, then you just need to plug-in the numbers into the formula). What you know almost surely, is that the forecasts are not independent: they are predicting the same thing, likely they are deterministic functions of the same, or similar, data, and if the forecasts are better then random, you would assume they give similar answers. Moreover, even if you made a completely unreasonable assumption that the forecasts are independent, the terms in Bayes theorem would cancel out for the independent events

$$ \require{cancel} P(A\mid B) = \frac{P(B \mid A) P(A)}{P(B)} = \frac{\frac{\cancel{P(A)} P(B)}{\cancel{P(A)}} P(A)}{P(B)} = \frac{\cancel{P(B)}P(A)}{\cancel{P(B)}} = P(A) $$

by the definition of independence.

Moreover, you don't have conditional probabilities, but rather the predictions and their accuracies, so what you can do, is you can take some form of average weighted by the accuracies, as described in the linked thread.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.