5
$\begingroup$

The title is a mouthful, but here is what it amounts to:

Under a proposal distribution using an independent chain, the probability of jumping to point $x$ is independent of the current position $y$ of the chain. Thus $q(x; y) = g(x)$. Now suppose I choose an asymmetric continuous distribution for $g(y)$, for example log-normal. In such cases my acceptance function $\rho(x;y)$ becomes

$\rho(x;y)=\min \left( \frac{f(x)}{f(y)}*\frac{q(y|x)}{q(x|y)},1 \right)$

Now what I don't understand is how to compute $q(y|x)=P(x \rightarrow y) = P(y)$ and $q(x|y)=P(y \rightarrow x) = P(x)$. Since $y$ and $x$ are drawn from continuous densities, their probabilities are zero. I must be wrong somewhere in my reasoning, but do not know where. Anyone can clarify?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Given Juho's answer, I think I confused the density $q(x)$ with probability $P(X=x)$. The acceptance is based on the density function, which is not zero for all $x \in X$. But then it seems my assumption $q(x|y)=P(y \rightarrow x) = P(X=x)$ is wrong? $\endgroup$
    – Davor Josipovic
    Apr 19, 2016 at 5:37

2 Answers 2

Reset to default
2
$\begingroup$

When the (conditional) proposal distributions are continuous, you compute the ratio of the densities,

\begin{equation} \frac{q(y\mid x)}{q(x \mid y)} = \frac{g(y)}{g(x)}. \end{equation}

Note also that the relevant asymmetry for needing to compute the proposal ratio (instead of plain Metropolis where the proposal ratio is $1$) is whether $q(y\mid x)=q(x\mid y)$. In the independent case, this amounts to $g(x)=g(y)$. So, you would need to compute the aforementioned ratio of proposal densities even if you used a symmetric distribution (such as normal). The only independent proposal case where the ratio disappears is a uniform density.

Improve this answer
$\endgroup$
2
$\begingroup$

All the terms used in a Metropolis-Hastings ratio are densities. This means densities against the same dominating measure, which most often is Lebesgue. Hence, with your notations, assuming all distributions are absolutely continuous against the Lebesque measure, $q(x|y)$ denotes a conditional density in $x$ given the realisation $y$ of the random variable $Y$.

In more complex settings when the proposal and the target are not continuous against the Lebesgue measure, if $\Pi$ is the target measure and $P(\cdot|y)$ the conditional proposal, the Metropolis-Hastings ratio writes $$1 \wedge \dfrac{\dfrac{\text{d}P(\cdot|x^\text{new})}{\text{d}\Pi}(x^\text{old})}{\dfrac{\text{d}P(\cdot|x^\text{old})}{\text{d}\Pi}(x^\text{new})}$$ expressing the conditional distributions in terms of their density against the target measure.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ What if the conditional proposal $P(\cdot|y)$ does not have a density with respect to the target measure $\Pi$? For example, $\Pi$ is a Gamma distribution and $P$ is Gaussian drif. $\endgroup$
    – jII
    Nov 13 at 15:49
  • $\begingroup$ @jII: besides the typo, what do you mean by Gaussian drif(t)? $\endgroup$
    – Xi'an
    Nov 13 at 21:13
  • $\begingroup$ E.g., $P(\cdot |y) = \mathrm{Normal}(y, 1)$ $\endgroup$
    – jII
    Nov 13 at 23:45
  • $\begingroup$ I was wondering if you were talking of a diffusion process. Since the Gamma and Normal distributions are ac wrt the Lebesgue measure, there is no issue in this case. $\endgroup$
    – Xi'an
    Nov 14 at 7:49
  • $\begingroup$ It seems the general theory (which unifies both the expression in your post and the usual expression using ratios of densities w.r.t Lebesgue measure) is as described in Andrieu et al. (2021; Theorem 3) [arxiv.org/pdf/2012.14881.pdf]. Neklyodov et al. (2020) discuss implications of these generalized ratios (using a less measure-theoretic approach) for unifying many other MCMC algorithms besides MH [arxiv.org/pdf/2006.16653.pdf]. $\endgroup$
    – jII
    Nov 14 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.