Variance of Rao Blackwellization for Monte Carlo Estimate of Expectation from https://arxiv.org/pdf/1401.0118.pdf
If we have a function $J(X,Y)$ of two random variables $X$ and $Y$ and we want to compute the expectation $\mathbb E_{p(X,Y)}[J(X,Y)]$.
We define $\hat J(X)= \mathbb E_{p(Y)}[J(X,Y)\mid X]$. 
Note that: $$\mathbb E_{p(X,Y)}[J(X,Y)] = \mathbb E_{p(X)} [\hat J(X)]$$
So we can use $\hat J(X)$ instead of $J(X, Y)$ in a Monte-Carlo Estimate. 
For the variance holds (variance reduction)
$$
var(\hat J(X)) = var(J(X,Y)) - \mathcal E[(J(X,Y) - \hat J(X))^2]
$$
Here I have two questions:


*

*How can we proof that (variance reduction)? In the paper no proof is given.


Thanks.
 A: The variance reduction follows from the the law of total variance. Suppose that $W,Z$ are two random variables, then it follows that
$$
\mathbb{V}(W)=\mathbb{V}(\mathbb{E}(W\vert Z))+\mathbb{E}(\mathbb{V}(W\vert Z))
$$
then, replace $W$ by $J(X,Y)$ and $\mathbb{E}(W\vert Z)$ by $\hat{J}(X)$ and we obtain:
$$
\mathbb{V}(J(X,Y))=\mathbb{V}(\hat{J}(X))+\mathbb{E}(\mathbb{V}(J(X,Y)\vert X))
$$
Notice that the second summand on the right hand side is given by
$$\mathbb{V}(J(X,Y)\vert X)=\mathbb{E}(J(X,Y)^2\vert X)-(\mathbb{E}J(X,Y)\vert X)^2=\mathbb{E}(J(X,Y)^2\vert X)-\hat{J}(X)^2
$$
plug into the ANOVA identity, solve with respect to $\mathbb{V}(\hat{J}(X)$ to obtain
$$
\mathbb{V}(\hat{J}(X)=\mathbb{V}(J(X,Y))-\left(\mathbb{E}(J(X,Y)^2)-\mathbb{E}(\hat{J}(X)^2)\right)=\mathbb{V}(J(X,Y))-\mathbb{E}\left(\left(J(X,Y)-\hat{J}(X)\right)^2\right)
$$
as desired.
Hope this helps.
A: I have not read the paper but the variance reduction provided by an iid sample is obvious from the variance Pythagorean decomposition and less obvious for a two-stage Gibbs sampler (Liu, Wong & Kong, 1994). For a general MCMC chain there is no reason to see variance reduction due to the correlation between the elements of the chain (Liu, Wong & Kong, 1995).
