throwing away all Gibbs samples after approximation

Question

This is more of a theory question, consider:

$$P(w_1|D)=\int P(w_1|S)P(S|D)d(S)$$

which we approximate via Gibbs sampling $S$ (assume the initial state of the Gibbs sampler is denoted by $M_0$), instead of computing the integral directly. Now,

• assume for a particular $w$, i.e. $w_1$, we have $N$ samples for $S$. What does it mean if we throw away these samples after evaluating $P(w_1|D)$, and when we get $w_2$, we start the sampler from $M_0$ again. Does it still construct a valid sampler?

• Why do this? Well, assume the size of the graphical model is so large that keeping samples is not practical (memory wise).

Answer 1

After reading the further explanations in the comments (which should definitely be moved to the question itself), I feel that using the same starting point $M_0$ for the MCMC chain is not necessarily damaging to the approximation if

1. A large enough burn-in (or warm-up) chunk of the simulations is not used for the approximation;
2. The total number of MCMC simulations is such that convergence (up to some degree of precision) is guaranteed.

My reasoning is the same as with regular Monte Carlo approximations to functions. Using the same simulations for different entries of the function is reducing the variability of the approximation and produces smoother versions of the approximated functions.

As we wrote in our book Introducing Monte Carlo Methods with R (2010, Section 5.4.1),

If $h(x)$ can be written as $\mathbb{E}[H(x,Z)]$ but is not directly computable, a natural Monte Carlo approximation of $h$ is $$ \hat h(x) = {1\over m} \sum_{i=1}^m H(x,z_i),$$ where the $Z_i$'s are generated from the conditional distribution $f(z|x)$. This approximation yields a convergent estimator of $h(x)$ for every value of $x$ (that is, it provides a pointwise convergent estimator), but its use in optimization setups is not recommended because, since the sample of $Z_i$'s changes with every value of $x$, using an iterative optimization algorithm over the $x$'s will result in an unstable sequence of evaluations of $h$ and thus in a rather noisy resolution to $\arg\max h(x)$.

Answer 2

Actually I guess the clarification doesn't really change my answer. If you keep re-starting your chain from the same position, this would violate the detailed balance condition: $$ q(x|y)\pi(y)=q(y|x)\pi(x) $$ Where $q$ is the proposal distribution and $\pi$ the target distribution. The reason being that $q(M_0|y)\pi(y)$ is now certainly not equal to $q(y|M_0)\pi(M_0)$. That is, since you always keep going back to $M_0$ at certain points in your chain regardless of your current state, the probability of this particular transition is completely unrelated to the probability of the transition in the opposite direction (which is essentially what you need for detailed balance).

More intuitively, your chain just wouldn't mix and explore $\pi$ properly because you keep restarting it from the same (arbitrary) starting point.

Memory shouldn't be a problem here since, if you want to compute an expectation on $w$, you only ever need to store one state of $S$ (the most recent sample), which presumably occupies no more memory than storing $M_0$.

Edit: I assumed you ultimately wanted to draw samples of $w$ and thereby approximate $p(w|D)=\int p(w|S)p(S|D)ds$, but with a bit more clarification it now seems that you want to calculate the probability of each possible value of $w$ with separate Gibbs samplings. Is there a special reason why you want to do it this way? The way I described above is more standard (and simple to implement).