2
$\begingroup$

The Metropolis-Hastings algorithm is a generalization of the older Metropolis algorithm. As part of these algorithms, they compute a ratio called the Metropolis ratio:

$$ r = \frac{P(x')}{P(x)}\frac{g(x | x')}{g(x' | x)} $$

where $g()$ are the proposal distributions and $P()$ the transition probabilities (or likelihoods at the different trial parameter values, $x$ and $x'$). For the Metropolis algorithm, the $g()$'s are referred to as symmetrical (and therefore cancel out) whereas, in the Hastings variant, they are asymmetrical and don't cancel out.

Can someone explain in plain language what is the meaning of symmetric and asymmetric in this situation? I thought at one point it referred to the symmetry of the normal distribution used to draw the values but according to the answer below that is not the case. Most of the books and blogs I've read are very obscure on this matter.

And related, how does one compute the asymmetric ratio in code, eg Python?

I found this question: What is the deeper intuition behind the symmetric proposal distribution in the Metropolis-Hastings Algorithm?

that almost gives me a plain answer but not quite, the key sentence seems to be: "If one uses a Normal distribution not centered at the previous value in the Metropolis-Hastings proposal (as e.g. in the Langevin version), the Normal distribution remains symmetric as a distribution but the proposal distribution is no longer symmetric " but I'm not entirely sure what it means especially with reference to actually computing the asymmetric ratio.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Let $\theta_1$ and $\theta_{2}$ be two points in your parameter space. That the proposal is symmetric means just that the probability of proposing $\theta_{2}$ from $\theta_1$ is the same as the probability of proposing $\theta_1$ from $\theta_2$. $\endgroup$
    – baruuum
    Sep 7, 2019 at 0:26
  • $\begingroup$ ok, I see that, but how does one compute the two proposal distributions if they are not symmetric as required by the Hastings algorithm? $\endgroup$
    – rhody
    Sep 7, 2019 at 3:11
  • $\begingroup$ I just found this, I think it explains it for me: stats.stackexchange.com/questions/27913/…. Funny thing is the answer is in the question rather than the answers. $\endgroup$
    – rhody
    Sep 7, 2019 at 3:15
  • $\begingroup$ yes, it makes also clear why a normal distribution not centered at the "current position" for every step would be asymmetric. $\endgroup$
    – baruuum
    Sep 7, 2019 at 4:51

1 Answer 1

Reset to default
0
$\begingroup$

An example with little math: my current state is at my house. I have a 37% chance of proposing to go to the bus stop. But if I were at the bus stop, I have a 42% chance of proposing to go to my house. 37=/=42 hence asymmetric. Putting a bit of math back in, a proposal is symmetric if p(x'|x) = p(x|x') for all x.

Maybe a question would be: is a normal distribution always symmetric? No. Why not? Imagine if u designed a proposal to be N(x, x^2). This is not symmetric for every x.

Another question: why would I want this? I might want this because I want a proposal with high variance around flat regions of my posterior, and small variance around peaks. Imagine using gradients to help achieve this.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.