4
$\begingroup$

The convergence theorem for Gibbs sampling states:

Given a random vector $X$ with components $X_1,X_2,...X_K$ and the knowledge about the conditional distribution of $X_k$ we can find the actual distribution using Gibbs Sampling infinitly often.

The exact theorem as stated by book (Neural Networks and Learning Machines):

The random variable $X_k^{(n)}$ converges in distribution to the true probability distributions of $X_k$ for k=1,2,...,K as n approaches infinity

$\lim_{n \rightarrow \infty}P(X^{(n)}_k \leq x | X(0)) = P_{X_k}(x) $ for $k > = 1,2,...,K$

where $P_{X_k}(x)$ is the marginal cumulative distribution function of $X_k$

While doing research on this, for a deeper understanding, I ran across this answer. Which explains quite well how to pick a single sample using the Method, but I am not able to extend/modify it to fit the convergence theorem, as the result of the given example is one sample (spell) and not a final/actual probability distribution.

Therefore, how do I have to modify that example to fit the convergence theorem?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ That doesn't look like a convergence theorem to me... convergence theorems generally say something like "as $t \to \infty$, some function of $t$ (e.g., a probability distribution) approaches some other function". $\endgroup$
    – jbowman
    Commented Oct 10, 2013 at 21:02
  • 1
    $\begingroup$ Just because you have a Gibbs sampler doesn't guarantee convergence (it's quite easy to construct a situation where sampling the full conditionals won't converge). Convergence of a particular implementation of Gibbs sampling (that is, with a particular model), and for MCMC implementations in general, is shown by establishing that the sampling scheme satisfies the conditions for convergence of a Markov Chain (usually fairly easy). If you look at what conditions have to apply for a Markov Chain to converge to its stationary distribution, you can see what you need to hold. $\endgroup$
    – Glen_b
    Commented Oct 10, 2013 at 21:29
  • $\begingroup$ I added the quotation from the book I am using $\endgroup$
    – Sim
    Commented Oct 10, 2013 at 21:36

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.