I aim at improving my understanding of Bayesian model averaging in the context of predictions and I am somewhat stuck on how to implement this numerically. Lets say I have two models $\mathcal{M}_1$ and $\mathcal{M}_2$, containing likelihood-functions $\mathcal{L}_i(y_t|\Theta_i)$ and priors for the parameters $\Theta_i$ for the time-series of interest $y_t$. Furthermore I have prior beliefs regarding the models, denoted as $\pi(\mathcal{M}_k)$. After observing $D=(y_1,\ldots,y_T)$ I would like to compute predictions $y_{T+1}$ by evaluating the posterior predictive of the two models. In my understanding this takes the form: $$p(y_{T+1}|D)=\sum_{k=1}^2 \underbrace{p(y_{T+1}|D,\mathcal{M}_k)}_\text{MCMC draws}\underbrace{p(\mathcal{M}_k|D)}_?$$ Given MCMC draws $\{\tilde{y}_{T+1} ^{(i)}\}_{\mathcal{M}_i}$ from the posterior predictive for each model and $p(\mathcal{M}_k|D)$ I would approximate $p(y_{T+1}|D)$ by averaging the draws, weighting with $p(\mathcal{M}_k|D)$ .

According to Bayes Theorem I know that $$p(\mathcal{M}_k|D)\propto p(D|\mathcal{M}_k)\pi(\mathcal{M}_k)$$ where $$p(D|\mathcal{M}_k)=\int p(D|\Theta_k,\mathcal{M}_k)p(\Theta_k|\mathcal{M}_k)d\Theta_k \\ \approx \sum_{i=1}^M p(D|\Theta_k ^{(i)},\mathcal{M}_k)$$ given $M$ MCMC draws from the posterior distribution of the parameters $\Theta_k$.

My question is: Is this correct so far? And if yes, would it be correct to compute $p(D|\mathcal{M}_k)$ based on the approximation first and to weight the draws from the posterior predictive in the next step accordingly?


Your approximation$$(D|\mathcal{M}_k)=\int p(D|\Theta_k,\mathcal{M}_k)p(\Theta_k|\mathcal{M}_k)d\Theta_k \\ \approx \sum_{i=1}^M p(D|\Theta_k ^{(i)},\mathcal{M}_k)$$is alas incorrect when the $\Theta_k^{(i)}$ are generated from the posterior by MCMC. They should be generated from the prior! (There is a $1/M$ missing as well but this is a typo.) This error can be found repeatedly in the literature, as we exposed in a Bayesian Analysis paper with Jean-Michel Marin discussing similar proposals by Scott (2002) and Congdon (2006).

The core notion is that there is no simple efficient way to estimate the posterior probability of a model based solely on separate simulations from different priors. Reversible jump MCMC operates because it jumps between models and jumps at random.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.