Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Homework question:

Consider the 1-d Ising model.

Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1

$\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$

Design a gibbs sampling algorithm to generate samples approximately from target distribution $\pi(x)$.

My attempt:

Randomly choose values (either -1 or 1) to fill vector $x = (x_1,...x_{40})$. So maybe $x = (-1, -1, 1, 1, 1, -1, 1, 1,...,1)$. So this is $x^0$.

So now we need to move on and do the first iteration. We have to draw the 40 different x's for $x^1$ separately. So...

Draw $x_1^1$ from $\pi(x_1 | x_2^0,...,x_{40}^0)$

Draw $x_2^1$ from $\pi(x_2 | x_1^1, x_3^0,...,x_{40}^0)$

Draw $x_3^1$ from $\pi(x_3 | x_1^1, x_2^1, x_4^0,...,x_{40}^0)$


So the part that's tripping me up is how do we actually draw from the conditional distribution. How does $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ come into play? Maybe an example of one draw would clear things up.

share|improve this question
up vote 5 down vote accepted

Look at this case first. Dropping terms that do not depend on $x_1$, we have. $$ \pi(x_1\mid x_2,\dots,x_d) = \frac{\pi(x_1,x_2,\dots,x_d)}{\pi(x_2,\dots,x_d)} \propto e^{x_1 x_2} $$ $$ P(X_1=-1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{-x_2}}{C} $$ $$ P(X_1=1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{x_2}}{C} $$ $$ \frac{e^{-x_2}}{C} + \frac{e^{x_2}}{C} = 1 \Rightarrow C = 2 \cosh x_2 $$

x_1 <- sample(c(-1, 1), 1, prob = c(exp(-x_2), exp(x_2)) / (2*cosh(x_2)))

Generalize it to $x_2,\dots,x_{40}$ (take notice of the differences; see Ilmari's comment bellow).

Finally, go beyond the programming homework and learn more about the Ising model. It is very important historically. Can you use Ising's analytic results to check your simulation? What was Ising trying to see? Anything related to phase transitions? Did he get the kind of behavior that he expected? How does it relate to your own (I mean you) experience with properties of ferromagnetic materials? Does a two-dimensional model display the same behavior (Onsager)? What happens if you increase the "range" of the interactions (Dyson) in the one-dimensional model?

share|improve this answer
So, it ends up only being dependent on the value immediately before it in the vector i.e. the only term that depends on $x_1$ is $x_2$, the only term that depends on $x_{23}$ is $x_{24}$, etc. What about the case of $x_{40}$? How do we draw it since the conditional distribution only seems to apply for $i=1$ through 39? – user2079802 Apr 23 '14 at 3:16
@user2079802: No, for $x_2$ through $x_{39}$ you get two terms in the exponent: $\pi(x_i\mid x_1,\dotsc,x_{i-1}; x_{i+1},\dotsc,x_d)$ $\propto$ $\exp(x_{i-1} x_i+x_i x_{i+1})$. But it's still easy enough to evaluate that for $x_i=\pm1$. – Ilmari Karonen Apr 23 '14 at 5:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.