# Variance of Rao Blackwellization for Monte Carlo Estimate of Expectation

If we have a function $$J(X,Y)$$ of two random variables $$X$$ and $$Y$$ and we want to compute the expectation $$\mathbb E_{p(X,Y)}[J(X,Y)]$$.

We define $$\hat J(X)= \mathbb E_{p(Y)}[J(X,Y)\mid X]$$.

Note that: $$\mathbb E_{p(X,Y)}[J(X,Y)] = \mathbb E_{p(X)} [\hat J(X)]$$

So we can use $$\hat J(X)$$ instead of $$J(X, Y)$$ in a Monte-Carlo Estimate.

For the variance holds (variance reduction) $$var(\hat J(X)) = var(J(X,Y)) - \mathcal E[(J(X,Y) - \hat J(X))^2]$$

Here I have two questions:

• How can we proof that (variance reduction)? In the paper no proof is given.

Thanks.

• can you be a bit more specific about your second question so I can improve my answer to your question? Dec 11 '18 at 21:02
• Thank you very much. I finally got it. So the second question is not important any more. Dec 14 '18 at 16:35

The variance reduction follows from the the law of total variance. Suppose that $$W,Z$$ are two random variables, then it follows that $$\mathbb{V}(W)=\mathbb{V}(\mathbb{E}(W\vert Z))+\mathbb{E}(\mathbb{V}(W\vert Z))$$ then, replace $$W$$ by $$J(X,Y)$$ and $$\mathbb{E}(W\vert Z)$$ by $$\hat{J}(X)$$ and we obtain: $$\mathbb{V}(J(X,Y))=\mathbb{V}(\hat{J}(X))+\mathbb{E}(\mathbb{V}(J(X,Y)\vert X))$$ Notice that the second summand on the right hand side is given by $$\mathbb{V}(J(X,Y)\vert X)=\mathbb{E}(J(X,Y)^2\vert X)-(\mathbb{E}J(X,Y)\vert X)^2=\mathbb{E}(J(X,Y)^2\vert X)-\hat{J}(X)^2$$ plug into the ANOVA identity, solve with respect to $$\mathbb{V}(\hat{J}(X)$$ to obtain $$\mathbb{V}(\hat{J}(X)=\mathbb{V}(J(X,Y))-\left(\mathbb{E}(J(X,Y)^2)-\mathbb{E}(\hat{J}(X)^2)\right)=\mathbb{V}(J(X,Y))-\mathbb{E}\left(\left(J(X,Y)-\hat{J}(X)\right)^2\right)$$ as desired.