Show that the total variation distance of the Metropolis kernel to its proposal kernel is equal to the rejection probability

Question

Furthermore, let

• $(E,\mathcal E,\lambda)$ be a $\sigma$-finite measure space;
• $Q$ be a Markov kernel on $(E,\mathcal E)$ with density $q$ with respect to $\lambda$;
• $\mu$ be a probability measure on $(E,\mathcal E)$ with density $p$ with respect to $\lambda$
• $$\alpha(x,y):=\left.\begin{cases}\min\left(1,\displaystyle\frac{p(y)q(y,x)}{p(x)q(x,y)}\right)&\text{, if }p(x)q(x,y)>0\\1&\text{, otherwise}\end{cases}\right\}\;\;\;\text{for }x,y\in E$$
• $$r(x):=1-\int Q(x,{\rm d}y)\alpha(x,y)\;\;\;\text{for }x\in E;$$
• $\kappa$ denote the transition kernel of the Metropolis-Hastings algorithm with proposal kernel $Q$ and target distribution $\mu$; i.e. $$\kappa(x,B)=\underbrace{\int_BQ(x,{\rm d}y)\alpha(x,y)}_{=:\;\pi(x,\;B)}+r(x)1_B(x)\;\;\;\text{for all }(x,B)\in E\times\mathcal E.$$

In Proposition 5.6.3 of this paper, it is claimed that, if $x\in E$ with $p(x)>0$, then $$\left\|\kappa(x,\;\cdot\;)-Q(x,\;\cdot\;)\right\|=r(x),\tag1$$ where $\|\;\cdot\|$ denotes the total variation norm. How do we proof this result?

A proof is given, but I don't understand it. The paper is considering MALA, instead of the general setup here, but I don't think this is crucial. I'm willing to impose further assumptions on $Q$, if necessary.

If $\nu_i$ is a probability measure on $(E,\mathcal E)$, the total variation distance between $\nu_1$ and $\nu_2$ is defined as $$\|\nu_1-\nu_2\|:=\sup_{B\in\mathcal E}(\nu_1-\nu_2)(B).$$

Now, what I get is that $$\kappa(x,B)-Q(x,B)=\begin{cases}Q(x,B^c)-\pi(x,B^c)&\text{, if }x\in B;\\\pi(x,B)-Q(x,B)&\text{, otherwise}\end{cases}\tag2$$ for all $B\in\mathcal E$.

For $B=\{x\}$, we obtain $$\kappa(x,B)-Q(x,B)=r(x)\tag3.$$ So, the question reduces to why we maximize $(2)$ by choosing $B=\{x\}$?

For this $B$, we clearly got $x\in B$ and hence $\kappa(x,B)-Q(x,B)=Q(x,B^c)-\pi(x,B^c)$. Clearly, the first summand $Q(x,B^c)$ will increase when we reduce $B$. But it's not clear to me, why this increase is larger than the decrease from $\pi(x,B^c)$. Seems like we need some kind of monotonicity between $Q$ and $\pi$ here ...

Answer 1

Conditional on $x$, the "coupling" interpretation is to start with the move attached to $Q(x,\cdot)$ and to propose to couple the output with $\kappa(x,\cdot)$, a coupling that is accepted with probability $1-r(x)$. Hence the total variation distance between $Q(x,\cdot)$ and $\kappa(x,\cdot)$ is zero with probability $1-r(x)$. And $1$ (i.e., one) with probability $r(x)$ since $$\mathbb I_{\{x\}^c}(x) - Q(x,\{x\}^c)=1$$ if one assumes that $Q$ is free of atoms.