If N Bayesians all estimate the probability of some event, how do you aggregate their estimates to get a final estimate? (Assume they are all equally trustworthy, or at least you have no reason to favor one over another)

For example, if one says 50%, another 80%, and a third 70%, do you take the mean, median, geometric mean?

  • 3
    $\begingroup$ What data are they using and what priors are they using? $\endgroup$ – whuber Sep 1 '18 at 21:10
  • 1
    $\begingroup$ The problem has no Bayesian answer when the $N$ priors are distinct. As a Bayesian you have to construct a new prior based on your prior information and beliefs, for instance the uniform mixture of all $N$ priors. $\endgroup$ – Xi'an Sep 2 '18 at 3:56
  • $\begingroup$ It can be argued that under realistic conditions, bayesians will have common priors -- see Uncommon Priors Require Origin Disputes -- paywalled but the preview gives a convincing outline. $\endgroup$ – shimao Sep 2 '18 at 21:31

Your question is presently ill-posed, and it really depends on what kind of "aggregation" you have in mind. Each Bayesian has a legitimate posterior estimate based on observed data and a prior belief, so presumably if you want "aggregate" these posterior estimates, you want to use a prior that is related somehow to the priors of the Bayesians in your question (e.g., a combination of their priors). One way to specify the "aggregation" is to specify how your own prior relates to the priors of these Bayesians, which in turn determines how your posterior relates to their posteriors. I will show you how you can do this below.

Since you have not specified to the contrary, I am going to assume that all the Bayesians in your question observe the same data $\mathbf{x}$, which they all take to come from the same sampling distribution with likelihood function $L(\theta)$. I will also assume that, conditional on the unknown parameter $\theta \in \Theta$, the probability of the event of interest is $0 \leqslant h(\theta) \leqslant 1$. This setup means that all the Bayesians perform the same inference, but they may each have different priors for the unknown parameter. (And hence, their estimated probabilities of the event may differ.)

Now, suppose that your Bayesians have priors $\pi_1,...,\pi_N$. Then for Bayesian $i$ the posterior probability of the event of interest is:

$$H_i = \mathbb{E}(h(\theta)| \mathbf{x},\pi_i) = \frac{\int_\Theta h(\theta) L(\theta) \pi_i(\theta) d\theta}{\int_\Theta L(\theta) \pi_i(\theta) d\theta}.$$

Now, how you aggregate these probabilities depends on how your own prior relates to the priors of the $N$ Bayesians in the analysis. For example, if your prior is some convex combination of priors of these Bayesians, then you have $\pi_* = \sum_{i=1}^n \lambda_i \pi_i$ and your corresponding posterior probability is:

$$H_* = \frac{\int_\Theta h(\theta) L(\theta) \pi_*(\theta) d\theta}{\int_\Theta L(\theta) \pi_*(\theta) d\theta} = \frac{\sum_{i=1}^n \lambda_i \int_\Theta h(\theta) L(\theta) \pi_i(\theta) d\theta}{\sum_{i=1}^n \lambda_i \int_\Theta L(\theta) \pi_i(\theta) d\theta} = \sum_{i=1}^n \lambda_i^* H_i,$$

where the updated posterior weights $\lambda_1^*,...,\lambda_N^*$ are given by:

$$\lambda_i^* = \frac{\lambda_i \int_\Theta L(\theta) \pi_i(\theta) d\theta}{\sum_{i=1}^n \lambda_i \int_\Theta L(\theta) \pi_i(\theta) d\theta}.$$

This gives you a rule for "aggregating" the posterior estimates of your Bayesians, based on an assumption that your preferred prior is some combination of their priors. As you can see, the updated posterior weightings are related to the prior weightings by a linear weighting that depends on the likelihood and priors. In the case where the Bayesians are "equally trustworthy" you might decide that your prior is a simple average of their priors,so that $\lambda_1 = ... = \lambda_N = \tfrac{1}{N}$. In this special case, the posterior weightings for the estimated probability of the event are:

$$\lambda_i^* = \frac{\int_\Theta L(\theta) \pi_i(\theta) d\theta}{\sum_{i=1}^n \int_\Theta L(\theta) \pi_i(\theta) d\theta}.$$

This result could arguably meet your requirement to "aggregate" the posterior estimates based on each Bayesian being "equally trustworthy". As you can see, even with the simplification of equal weighting on the priors, the aggregation is still more complicated than taking a simple mean, median or geometric mean.

  • $\begingroup$ How does this compare to Bayesian model averaging? For instance, could one frame the question of averaging several posterior estimates arising from different hyperparameters for a prior as a BMA problem? $\endgroup$ – user79097 Mar 11 at 7:13
  • $\begingroup$ I don't really see why not - this weighting form would be applicable whenever you have a prior that is a convex combination of sub-priors, so it would arise in model averaging so long as that averaging is an average over priors. $\endgroup$ – Ben Mar 11 at 7:30
  • $\begingroup$ Thanks for your helpful answer! Since I still have a few conceptual questions, I have made my concern(s) more precise in this thread: stats.stackexchange.com/questions/396799/… $\endgroup$ – user79097 Mar 11 at 7:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.