# Understanding Tierney Gibbs Sampler Kernel

In subsection 2.2 Tierney (1994) states:

Suppose $$X \sim \pi$$ and $$f$$ is a function on the state space $$E$$. Then the kernel $$P(x, A) = P(X\in A \mid f(X) = f(x))$$ leaves $$\pi$$ invariant.

However doesn't seem to provide any intuition or proof as to why that's the case. I know that for the kernel to leave $$\pi$$ invariant we must have $$\pi = \pi P \qquad \qquad \pi(A) = \int P(x, A)\, d\pi(x) = \int Q(f(x), A) \, d \pi(x)$$ How do we show this and what's the intuition behind this?

# Edit

Following comments here is my attempt. I will try to use the Law of Total Expectation. First, we can write $$\mathbb{E}_{\pi}[P(x, A)] = \mathbb{E}_\pi[P(X \in A \mid Y = y)]$$ By the TOWER property we know that if $$\nu$$ is the distribution for $$y$$ then $$\mathbb{E}_\nu[\mathbb{E}_\pi[P(X \in A \mid Y=y)] = P(X\in A) = \pi(A)$$ Now I only need to find what is the distribution $$\nu$$. Suppose $$\pi$$ is defined on $$E$$. Then $$f(E)$$ is image of the function $$f$$ and $$y \in f(E)$$. This means that our distribution $$\nu$$ must be defined on a sigma algebra of $$f(E)$$, for instance $$\sigma(f(E))$$ $$\nu: \sigma(f(E))\to [0, 1]$$ Since $$E = f^{-1}(f(E))$$ we can say $$\nu$$ is the pushforward of $$\pi$$ by $$f^{-1} \circ f$$ $$\nu = (f^{-1} \circ f)_* \pi = \pi$$ Therefore $$\int d\pi(x) \int P(X \in A\mid F(X) = f(x)) d\pi(x) = \pi(A)$$ ???

• You indeed need to prove the last equality holds. May 11, 2021 at 13:23
• The intuition is that the move is according to $\pi$ restricted to one level set of $f$. Integrating over $x$ recovers all level sets, May 11, 2021 at 13:46
• @Xi'an Thanks! I gave it a go but I didn't quite manage to prove it, do you mind having a look? May 11, 2021 at 15:05
• The expression$$\mathbb E_π[P(x,A)]=\mathbb E_π[P(X∈A∣Y=y)]$$should write$$\mathbb E_π[P(X,A)]=\mathbb E_{fπ}[P(X∈A∣Y)]$$as there is only one random variable left to integrate. May 12, 2021 at 6:48
• Note that the kernel is not of practical interest for MCMC, since the entire sequence of simulations remains at the same level set $f(x)=f(x_0)$. May 13, 2021 at 10:19

It looks like you have a good idea of the kind of tools you could use to prove this statement, but I was a bit unclear about your attempted proof, in particular what you meant by the measure/distribution $$\nu$$.
Intuitively, you compute $$P(X\in A) = \pi(A)$$ in two steps: first integrate across each level set of the function $$f$$, then integrate over all level sets of $$f$$; this is equivalent to integrating over the whole space $$E$$. You can see this most clearly in the case where $$f$$ takes on only $$1$$ or $$2$$ values. You could try working through these cases in detail to develop more intuition.
For the general case, it helps to identify $$P(X \in A \mid f(X)=f(x))$$ as an integral over a level set of $$f$$. In fact, $$P(X \in A \mid f(X)=f(x))=\mathbb{E}[1_{X\in A}\mid Y=f(x)]=\varphi(f(x))$$ where the function $$\varphi$$ is such that $$\varphi (Y)=\mathbb{E}[1_{X\in A}\mid Y]$$. (See e.g. Definition 8.24 in Probability Theory by Klenke for how this function is obtained. It's also discussed in this answer.)
By Theorem 4.10 (image measures) in the same book, or Lemma 6.12 in Probability with Martingales by Williams, we have that $$\int P(x, A)\, d\pi(x) =\int\varphi(f(x))\, d\pi(x)=\mathbb{E}[\varphi (f(X))]$$. This is just $$\mathbb{E}[\varphi(Y)]=\mathbb{E}[\mathbb{E}[1_{X\in A}\mid Y]]$$, which is equal to $$\pi(A)$$ by the law of total expectation.