Homework question:

Consider the 1-d Ising model.

Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1

$\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$

Design a gibbs sampling algorithm to generate samples approximately from target distribution $\pi(x)$.

My attempt:

Randomly choose values (either -1 or 1) to fill vector $x = (x_1,...x_{40})$. So maybe $x = (-1, -1, 1, 1, 1, -1, 1, 1,...,1)$. So this is $x^0$.

So now we need to move on and do the first iteration. We have to draw the 40 different x's for $x^1$ separately. So...

Draw $x_1^1$ from $\pi(x_1 | x_2^0,...,x_{40}^0)$

Draw $x_2^1$ from $\pi(x_2 | x_1^1, x_3^0,...,x_{40}^0)$

Draw $x_3^1$ from $\pi(x_3 | x_1^1, x_2^1, x_4^0,...,x_{40}^0)$


So the part that's tripping me up is how do we actually draw from the conditional distribution. How does $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ come into play? Maybe an example of one draw would clear things up.


Look at this case first. Dropping terms that do not depend on $x_1$, we have. $$ \pi(x_1\mid x_2,\dots,x_d) = \frac{\pi(x_1,x_2,\dots,x_d)}{\pi(x_2,\dots,x_d)} \propto e^{x_1 x_2} $$ $$ P(X_1=-1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{-x_2}}{C} $$ $$ P(X_1=1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{x_2}}{C} $$ $$ \frac{e^{-x_2}}{C} + \frac{e^{x_2}}{C} = 1 \Rightarrow C = 2 \cosh x_2 $$

x_1 <- sample(c(-1, 1), 1, prob = c(exp(-x_2), exp(x_2)) / (2*cosh(x_2)))

Generalize it to $x_2,\dots,x_{40}$ (take notice of the differences; see Ilmari's comment bellow).

Can you use Ising's analytic results to check your simulation?

  • $\begingroup$ So, it ends up only being dependent on the value immediately before it in the vector i.e. the only term that depends on $x_1$ is $x_2$, the only term that depends on $x_{23}$ is $x_{24}$, etc. What about the case of $x_{40}$? How do we draw it since the conditional distribution only seems to apply for $i=1$ through 39? $\endgroup$ – Collin Apr 23 '14 at 3:16
  • 1
    $\begingroup$ @user2079802: No, for $x_2$ through $x_{39}$ you get two terms in the exponent: $\pi(x_i\mid x_1,\dotsc,x_{i-1}; x_{i+1},\dotsc,x_d)$ $\propto$ $\exp(x_{i-1} x_i+x_i x_{i+1})$. But it's still easy enough to evaluate that for $x_i=\pm1$. $\endgroup$ – Ilmari Karonen Apr 23 '14 at 5:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.