2
$\begingroup$

I am trying to better understand better the Gelman/Rubin measure of convergence of MCMCs. The method starts off by defining two quantities: $B$ and $W$. $B$ is said to be the between chain variance (as typically $m$ chains are run in parallel), and $W$ is the average within chain variance. I understand $W$ - it is merely the average of all the variances in each of the individual chains. $B$, at first glance, appears to measure the average deviation across all chains of their average from the overall mean. However, $B$ is multiplied by $n$ - the sample size - which I don't quite understand. Texts say that this is because there are $n$ points in each chain, however, I was hoping someone could provide a little intuition here? Further, $W$ is combined with $B$ in a weighted average, with $W$'s weight being $(n-1)/n$, and $B$'s weight simply being $(1/n)$. This is then typically divided by $W$, and the square root taken. If $B$ equals $W$ I can see that this ratio is then $1$. However, I don't see how this ratio ever gets far away from $1$ for moderately large sample sizes. It must be the case that $B>>W$, part of which I can understand if the chains are initially overdispersed. Still, the fact that $B$ is multiplied by $(1/n)$, makes it look like this part of the weighted average should disappear at a fast rate, even if $B$ is large.

Basically, I think my question could be condensed into the following 'Why is this measure of convergence reasonable? Given that as $n$ gets large, ceteris paribus, the role played by the between chain variance appears to become smaller.'

$\endgroup$
  • $\begingroup$ There's an analogy to comparison of within-group and between-group mean squares in ANOVA. $\endgroup$ – Glen_b -Reinstate Monica Aug 25 '14 at 2:00
  • $\begingroup$ Hi Glen, thanks for that suggestion - I suspect that may allow me to answer why $B$ is multiplied by $n$. However, still the rest is unclear to me. $\endgroup$ – ben18785 Aug 25 '14 at 13:27
  • $\begingroup$ Not simply the formula, but the idea of what it's doing is analogous. If the chains are "more different" than might be explained by random variation, the chains cannot have converged to the same stationary distribution. So if between-chain-variation is higher than within-chain-variation suggests it should be, you'd conclude that the chains haven't really converged before the start of the measurement period. It's at least an intuitively plausible approach to measurement of non-convergence. $\endgroup$ – Glen_b -Reinstate Monica Aug 25 '14 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.