2
$\begingroup$

Like I undestand MCMC sampling, the fulfillment of the detailed balance equation guarantees that our MC has reached its stationary distribution (given we ensure ergodicity).

Detailed Balance is:

$\pi(x)q(x\rightarrow x')=\pi(x')q(x'\rightarrow x)$

with $\pi(x)$ being the probability to be in state $x$ and $q(x\rightarrow x')$ the transition probability from state $x$ to $x'$ at time $T$. (According to Russell, Stuart, and Peter Norvig. "AI a modern approach")

A problem I came across in MCMC is to find the right burn-in time $T$, the amount of samples needed to reach the stationary distribution. Why can we not use DBE as a stopping criterion? Why can we not compute whether DBE is fulfilled after each sample and then stop sampling as soon as it is fulfilled? Naively, it looks like $\pi(x)$ and $q(x\rightarrow x')$ could be computed emperically based on the samples obtained so far.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

the fulfillment of the detailed balance equation guarantees that our MC has reached its stationary distribution (given we ensure ergodicity).

This is not a correct statement. Detailed balance wrt $\pi(x)$ guarantees that $\pi(x)$ is the stationary distiribution for the Markov chain. The fact that the MC "reaches" this stationary distribution is the definition of ergodicity of the Markov chain. (Note that there is a difference between stationary distribution and limiting distribution).

Detailed balance is a property of a Markov chain and says close to nothing about the limiting behavior of the chain. So for a chain that is detailed balanced, the DBE will hold at every step of the Markov chain.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks! So, then DB is more a method to verify whether sampling from a particular MC would converge to some stationary distribution and so it can rule out some Markov chains as principally unable to represent the target distribution? $\endgroup$
    – dopexxx
    Jan 14, 2018 at 14:42
  • 1
    $\begingroup$ No, detailed balance does not guarantee convergence to the stationary distribution. DB is a property of a MC and detailed balance wrt $\pi(x)$ just gives you that $\pi(x)$ is the stationary distribution. The convergence to this stationary distribution depends on other properties being satisfied. There are many Markov chains that do not satisfy a detailed balance condition wrt $\pi$ but still have $\pi$ as the stationary distribution. For e.g., a three variable Gibbs sampler. $\endgroup$
    – Greenparker
    Jan 14, 2018 at 16:06
  • $\begingroup$ Okay, I phrased that question badly. I rather meant: If I want to sample from a complicated distribution $P(X)$ and I can come up with an ergodic MC and use DBE to show that the stationary distribution of the MC is $P(X)$, then I am lucky in the sense that sampling infinitely long from the MC will certainly yield samples from my target distribution (although I'm still left with my problem to find the burn-in time $T$). $\endgroup$
    – dopexxx
    Jan 17, 2018 at 10:23
  • $\begingroup$ @dopexxx Yes, that's a reasonable statement. $\endgroup$
    – Greenparker
    Jan 17, 2018 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.