I have a target distribution $\mu$ which I would like to investigate using, for instance Metropolis-Hastings-Green (MHG). So, given a Gaussian prior, $\pi$, and a likelihood $L$ such that $\mu(dx) \propto L(x) \pi(dx)$, I can proceed with the standard MHG with random walk proposals from the prior. But in the problems I'm interested in, I know I have multimodality in $\mu$, so I would like to use a multimodal proposal distribution, such as a mixture of two Gaussians, $\pi_1$ and $\pi_2$. Assuming I have likelihoods for each of these, $L_1$ and $L_2$, and I have chosen mixture weights, $w_1 + w_2 =1$, I can certainly sample from $w_1 \pi_1 + w_2\pi_2$. My question is that now I want to run MHG on top of this, and it's unclear to me how to set up the Metropolis ratio for accepting/rejecting the proposal. I've looked a bit in the literature, but haven't seen anything.

  • $\begingroup$ I don't understand the setting - there may be some notational confusion. In what sense do the components of the proposal have "likelihoods"? Can you give an example? $\endgroup$ – Juho Kokkala Oct 1 '16 at 10:29
  • 1
    $\begingroup$ Do you know $V(x)$ (Nonrandom? Can be evaluated at least pointwise?)? I suspect you are using the word prior in a nonstandard manner - if the target distribution arises from something else than a Bayesian model, I don't think there should be a "prior" or a "likelihood" involved at all. I also don't see why the mixture representation would complicate setting up the acceptance ratio (at least if $x$ lives in some $\mathbb{R}^n$ with fixed $n$)? Explicitly writing the $V(x)$, $\pi_1$ and $\pi_2$ for the particular case might help to clarify the question. $\endgroup$ – Juho Kokkala Oct 1 '16 at 20:15
  • 1
    $\begingroup$ What is $\pi_j$? Is your intended proposal $w_1\pi_1 + w_2\pi_2$, independent of the current state? And there are no other "components" in the setting? I don't understand what the likelihoods $L_1$ and $L_2$ mentioned in the question are. Also, if I understand the question correctly, I don't see any complications arising from the mixture - just plug in the densities? $\endgroup$ – Juho Kokkala Oct 2 '16 at 6:06
  • 1
    $\begingroup$ But I don't see why there are likelihoods and priors involved if your target is not a Bayesian posterior. Let alone why there are two "likelihoods". Are you somehow using "prior" to mean "proposal" and "likelihood" to mean "target / proposal"?. Also, why does the mixture proposal change anything - the acceptance ratio is computed just like with any other proposal - just use the density of the mixture (but this leads me to suspect I don't understand the question and you are trying to do something more complicated) $\endgroup$ – Juho Kokkala Oct 4 '16 at 6:19
  • 1
    $\begingroup$ I mean, as far as I understand the setting, your target has density $f(x) \propto \exp(-V(x))$ and your proposal (if it's meant to be independent of current state) has density $g(x_{new} \mid x_{old}) = w_1\,g_1(x_{new}) + w_2\,g_2(x_{new})$. These densities are all with respect to the same dominating measure (Lebesgue in $\mathbb{R}$ in this example)? The acceptance probability would then be simply $\min(1,~\frac{\exp(-V(x_{new})}{\exp(-V(x_{old})}\,\frac{w_1\,g_1(x_{old})+w_2\,g_2(x_{old})}{w_1\,g_1(x_{new})+w_2\,g_2(x_{new})})$. $\endgroup$ – Juho Kokkala Oct 4 '16 at 6:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.