Metropolis-Hastings - interpreting the transition kernel: alpha*proposal

Question

I thought I had great intuition and mathematical understanding of the Metropolis-Hastings algorithm, until closer inspection... as I started compiling my notes, I realized I do not understand the rejection step of the algorithm.

Here is what I understood:

We have a target distribution $\pi(x)$, and we construct a transition kernel $K(x' \mid x)$ such that the detailed balance equation holds. $$\pi(x)K(x' \mid x) = \pi(x') K(x \mid x')$$

We can choose $$K(x, x') = \displaystyle \alpha(x, x')q(x \mid x')$$ Where $\alpha$ is the Metropolis-Hastings ratio, and $q$ is some proposal distribution. This particular construction of $\alpha$ helps correct the discrepancies in our detailed balance equation, thus providing us flexibility in choosing $q$.

Where I am having problems:

• How do I think about $K$ as a distribution (or even visualize)? In particular, what is $\alpha(x, x')$?
• What's going on with the sampling step where we reject and stay at $x$? Originally I thought of it as some correction function, but the rejection meant $X' := X$ and thus instinctively, I want think of $K(X, X')$ as a mixture of a $\delta_{\{X\}}(X')$ and $q(X'|X)$, however, the mass associated with this dirac delta varies depending on $x'$...? Not quite a mixture model.
• Should I be looking to interpret $\alpha$ as some form of accept-reject algorithm?
• How do I write $K$ as a density?

Edit: Maybe this should be a question not a comment:

With regards to the order of derivations (ie. motivation), is the following a reasonable thought process?

1. We want to construct some transition kernel invariant to our target distribution.
2. We select some proposal distribution, and notice it breaks detailed balance equation.
3. we correct it with an acceptance-probability.
4. Due to this correction probability, we need to have some action corresponding to the complement accepting the proposed state -> we remain at our current state.

Question: Is this choice of "remain at current state" arbitrary?

Answer 1

When you write the Metropolis-Hastings density [wrt a dominating measure that is the sum of a measure $\text{d}\lambda$ that is absolutely continuous against the target and of a Dirac measure at $x$] as$$K(x, x') = \displaystyle \alpha(x, x')q(x \mid x')$$it should be $$K(x, x') = \displaystyle \alpha(x, x')q(x' \mid x)+ \int (1-\alpha(x, x'))q(x' \mid x)\text{d}x' \Bbb I_{x'=x}$$where $\Bbb I_{x'=x}$ denotes the indicator function, which corresponds to the part of the density that is for the Dirac mass at $x$. In particular, $\alpha(x, x')$ must satisfy [from the detailed balance equation] $$\pi(x)\alpha(x, x')q(x'\mid x)=\pi(x)\alpha(x´, x)q(x\mid x')\qquad 0\le\alpha(\cdot,\cdot)\le 1$$