4
$\begingroup$

In which situation would you refrain from using BMA. It seems to me it is always a good idea to use the posterior probabilities when infering/predicting.

$\endgroup$

1 Answer 1

2
$\begingroup$

If you have two candidate likelihoods $P_1(\text{data}=\{x_1\dots x_n\}|\theta_1)$ and $P_2(\text{data}=\{x_1\dots x_n\}|\theta_2)$ with two different parametrizations $\theta_1$ and $\theta_2$, the posteriors $\pi(\theta_1|\text{data})$ and $\pi(\theta_2|\text{data})$ will not be directly comparable. In such a case, "model averaging" on this level would be averaging apples $(\theta_1)$ and oranges $(\theta_2)$. So that is an example of a case where model averaging of posteriors is basically meaningless.

But if you mean to compute, say, Prob(data > $x_0$) where $x_0$ is some threshold value of data, then it could still make sense to use the posterior predictive distributions

$$ P_1(x|\text{data}) = \int P_1(x|\theta_1)\pi(\theta_1|\text{data}) d\theta_1 $$

$$ P_2(x|\text{data}) = \int P_2(x|\theta_2)\pi(\theta_2|\text{data}) d\theta_2 $$

and then find

$$ P_i=\text{Prob}_i(x>x_0|\text{data})=\int_{x>x_0} dx P_i(x|\text{data}), ~~~~i = 1, 2 $$ for each of the two models. You are then at liberty to average these numbers based on a prior preference $\phi_i, \phi_2, (\phi_1+\phi_2=1)$ for each model.

Here, model averaging is valid, but I would prefer a model selection process that settles on one model and doesn't require any model averaging. There should be some fundamental reason to prefer model 1 to model 2 $-$ unless the underlying observed phenomenon is itself a mixture process, which can happen, of course.


As a postscript: Jennifer Hoeting and et al have a quite readable tutorial on BMA. In section 4 Interpretation she cites David Draper as one who suggested the problematic apples-and-oranges aspect of BMA (nearly 30 years ago), but she defends BMA by adding a mixture hyperparameter (I call it $\phi$ above) that weights the various sub-models. Naturally, the hyperparameter needs a prior $\pi(\phi)$ and what would that be? Invoking the mixture and its prior seems to me to be like kicking the hard work of model selection down the road. It is not that it is wrong, per se, just that it isn't preferred, and by itself does not constitute a solution. There are no free passes to excuse you from doing good model selection.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.