Consider having two models, $$m_1$$ and $$m_2$$, for a set of data $$x$$, each model has associated parameters $$\theta_1$$, for model one, and $$\theta_2$$ for model two, (not necessarily the same dimension). I'd like to know something like the $$P(\theta,m|x)$$ given some data. I'm not really even sure how to write this as the parameters in the models may be completely different. So here are my thoughts as to how one might generate a sample from $$P(m, \theta|x)$$ using the Metropolis algorithm.

Simultaneously generate two independent MCMC chains, $$a$$ for $$\theta_1$$ from model 1 and $$b$$ for $$\theta_2$$ in model 2. Say these chains are of length $$n$$. In addition, generate a chain for a variable $$\mu$$ that stores your model selection,

1. generate a proposal, for the parameters in each model, $$a'$$ and $$b'$$, from a symmetric distribution
2. accept or reject the proposals using standard MCMC methods, leading to $$a_i$$ and $$b_i$$
3. propose $$\mu' \sim \text{Bernoulli}(0.8 \, \mu_{i-1} + 0.1)$$. Inother words, propose the same model as the previous iteration with prob 0.9, switch models with prob 0.1.
4. calculate an acceptance probability based on the likelihood ratio. Something like, let

$$L_{old} = \begin{cases} L(a_{i-1}|\mu_{i-1},x) & \mu_{i-1} = 0 \\ L(b_{i-1}|\mu_{i-1},x) & \mu_{i-1} = 1 \\ \end{cases},$$

and

$$L' = \begin{cases} L(a'|\mu',x) & \mu' = 0 \\ L(b'|\mu',x) & \mu' = 1 \\ \end{cases}$$

and define acceptance probability

$$p = \min\left\{ \frac{L' }{ L_{old} },1\right\}$$

1. Set $$\mu_i=\mu'$$ with probability $$p$$ and $$\mu_i=\mu_{i-1}$$ with probability $$1-p$$

In the end your $$\mu$$ maps you to which parameter set to use, the one in the chain for model 1, $$a$$, or model 2, $$b$$.

Is this sensible? Are there better ways to do this? My guess is that Ensamble modelling might be something to look into, but they are usually interested in prediction rather than the likelihood of parameters, based on my very limited understanding. Since the parameters may not be the same length, is there a way to modify $$p$$ so that it uses something equivalent to an AIC?

How to sample a joint posterior given multiple models?

Consider having two models, $$m_1$$ and $$m_2$$, for a set of data $$x$$, each model has associated parameters $$\theta_1$$, for model one, and $$\theta_2$$ for model two, (not necessarily the same dimension). I'd like to know something like the $$P(\theta,m|x)$$ given some data. I'm not really even sure how to write this as the parameters in the models may be completely different. So here are my thoughts as to how one might generate a sample from $$P(m, \theta|x)$$ using the Metropolis algorithm.

Simultaneously generate two independent MCMC chains, $$a$$ for $$\theta_1$$ from model 1 and $$b$$ for $$\theta_2$$ in model 2. Say these chains are of length $$n$$. In addition, generate a chain for a variable $$\mu$$ that stores your model selection,

1. generate a proposal, for the parameters in each model, $$a'$$ and $$b'$$, from a symmetric distribution
2. accept or reject the proposals using standard MCMC methods, leading to $$a_i$$ and $$b_i$$
3. propose $$\mu' \sim \text{Bernoulli}(0.8 \, \mu_{i-1} + 0.1)$$
4. calculate an acceptance probability based on the likelihood ratio. Something like, let

$$L_{old} = \begin{cases} L(a_{i-1}|\mu_{i-1},x) & \mu_{i-1} = 0 \\ L(b_{i-1}|\mu_{i-1},x) & \mu_{i-1} = 1 \\ \end{cases},$$

and

$$L' = \begin{cases} L(a'|\mu',x) & \mu' = 0 \\ L(b'|\mu',x) & \mu' = 1 \\ \end{cases}$$

and define acceptance probability

$$p = \min\left\{ \frac{L' }{ L_{old} },1\right\}$$

1. Set $$\mu_i=\mu'$$ with probability $$p$$ and $$\mu_i=\mu_{i-1}$$ with probability $$1-p$$

In the end your $$\mu$$ maps you to which parameter set to use, the one in the chain for model 1, $$a$$, or model 2, $$b$$.

Is this sensible? Are there better ways to do this? My guess is that Ensamble modelling might be something to look into, but they are usually interested in prediction rather than the likelihood of parameters, based on my very limited understanding. Since the parameters may not be the same length, is there a way to modify $$p$$ so that it uses something equivalent to an AIC?