2
$\begingroup$

I am just getting started with the Gibbs Sampler and came across an implementation from here and here and here. All of theses implementations are based on the first article.

There is an inner loop in the implementation and I don't understand it's purpose.

Here is the code (written in julia). It's been changed slightly to the implementation from the article, but not where it matters.

function gibbs(n, thin)

    #array to store the results
    mat = Array(Float64, (n,2))
    x = y = 0.0

    #outer loop number of samples to draw
    for i in 1:n

        #inner loop: purpose unknown. 
        for j in 1:thin
            x = rand ( Normal( .9 * y, 1 - .9^2))
            y = rand ( Normal( .9 * x, 1 - .9^2))
        end
        mat[i,1] = x; mat[i,2] = y
    end
    mat

#end of program
end

function main()

    gibbs(10000, 200)

end
main()       

From my understanding the inner loop creates an addition $n * thin$ amount of samples and thus decreases the likelihood of two consecutive draws being to close to each other. Is there another purpose to this?

$\endgroup$
  • 1
    $\begingroup$ It's to reduce the dependence in sampled values more generally (rather than necessarily being "near each other", though that's the typical case) $\endgroup$ – Glen_b -Reinstate Monica Sep 8 '15 at 15:40
  • $\begingroup$ Thanks for the answer. Care to post it as an answer then I can vote this question to answered. $\endgroup$ – Vincent Sep 8 '15 at 15:44
  • 1
    $\begingroup$ Fortunately you effectively have all parts of the answer I would have posted now, plus some more besides. All I can do is upvote them. $\endgroup$ – Glen_b -Reinstate Monica Sep 9 '15 at 4:40
4
$\begingroup$

As it was already mentioned, this is called thinning, however, as Kruschke noticed it is "rarely useful" and quoting Link and Eaton (2012), he writes that

... basic conclusion of the article is that thinning of chains is not usually appropriate when the goal is precision of estimates from an MCMC sample. (Thinning can be useful for other reasons, such as memory or time constraints in post-chain processing, but those are very different motivations than precision of estimation of the posterior distribution.)

The paper published in Methods in Ecology and Evolution is freely avilable from the publisher.

So it is disputable if thinning is that much helpful, i.e. the inner loop may not be that important in the code and in many cases can be simply omitted with saving the whole simulation output.

$\endgroup$
  • 2
    $\begingroup$ (+1) I came here to say precisely this! $\endgroup$ – Reinstate Monica Sep 8 '15 at 18:02
  • 2
    $\begingroup$ +1 I have to say that in most cases I agree; to thin enough to make it worthwhile means throwing out an enormous amount of information quite needlessly. $\endgroup$ – Glen_b -Reinstate Monica Sep 9 '15 at 4:38
4
$\begingroup$

Just to expand on what you and @Glen_b said: The point of the inner loop is to reduce correlation among sampled values. A common way to do this is by only taking every nth value, a process called Thinning.

From Wikipedia:

It is common to ignore some number of samples at the beginning and then consider only every th sample when averaging values to compute an expectation. … The reason for this is that (1) successive samples are not independent of each other but form a Markov chain with some amount of correlation; (2) the stationary distribution of the Markov chain is the desired joint distribution over the variables, but it may take a while for that stationary distribution to be reached.

Your example (the inner loop) does this slightly differently. Instead of reducing correlation between samples by taking every nth value, it obscures it by randomizing each sample thin number of times.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.