Skip to main content
1 of 3
Greenparker
  • 16k
  • 3
  • 51
  • 99

Assuming I take the mean of the posterior distribution rather than a random sample from it, is this what is commonly referred to as Rao-Blackwellization?

I am not very familiar with stochastic volatility models, but I do know that in most settings, the reason we choose Gibbs or M-H algorithms to draw from the posterior, is because we don't know the posterior. Often we want to estimate the posterior mean, and since we don't know the posterior mean, we draw samples from the posterior and estimate it using the sample mean. So, I am not sure how you will be able to take the mean from the posterior distribution.

Instead the Rao-Blackwellized estimator depends o the knowledge of the mean of the full conditional; but even then sampling is still required. I explain more below.

Suppose the posterior distribution is defined on two variables, $\theta = (\mu, \phi$), such that you want to estimate the posterior mean: $E[\theta \mid \text{data}]$. Now, if a Gibbs sampler was available you could run that or run a M-H algorithm to sample from the posterior.

If you can run a Gibbs sampler, then you know $f(\phi \mid \mu, data)$ in closed form and you know the mean of this distribution. Let that mean be $\phi^*$. Note that $\phi^*$ is a function of $\mu$ and the data.

This also means that you can integrate out $\phi$ from the posterior, so the marginal posterior of $\mu$ is $f(\mu \mid data)$ (this is not known completely, but known upto a constant). You now want to now run a Markov chain such that $f(\mu \mid data)$ is the invariant distribution, and you obtain samples from this marginal posterior. The question is

How can you now estimate the posterior mean of $\phi$ using only these samples from the marginal posterior of $\mu$?

This is done via Rao-Blackwellization.

\begin{align*} E[\phi \mid data]& = \int \phi \; f(\mu, \phi \mid data) d\mu \, d\phi\\ & = \int \phi \; f(\phi \mid \mu, data) f(\mu \mid data) d\mu \, d\phi\\ & = \int \phi^* f(\mu \mid data) d\mu \end{align*}

Thus suppose we have obtained samples $X_1, X_2, \dots X_N$ from the marginal posterior of $\mu$. Then $$ \hat{\phi} = \dfrac{1}{N} \sum_{i=1}^{N} \phi^*(X_i) $$

is called the Rao-Blackwellized estimator for $\phi$.

Greenparker
  • 16k
  • 3
  • 51
  • 99