Gibbs sampler implementation

Question

I am just getting started with the Gibbs Sampler and came across an implementation from here and here and here. All of theses implementations are based on the first article.

There is an inner loop in the implementation and I don't understand it's purpose.

Here is the code (written in julia). It's been changed slightly to the implementation from the article, but not where it matters.

function gibbs(n, thin)

    #array to store the results
    mat = Array(Float64, (n,2))
    x = y = 0.0

    #outer loop number of samples to draw
    for i in 1:n

        #inner loop: purpose unknown. 
        for j in 1:thin
            x = rand ( Normal( .9 * y, 1 - .9^2))
            y = rand ( Normal( .9 * x, 1 - .9^2))
        end
        mat[i,1] = x; mat[i,2] = y
    end
    mat

#end of program
end

function main()

    gibbs(10000, 200)

end
main()       

From my understanding the inner loop creates an addition $n * thin$ amount of samples and thus decreases the likelihood of two consecutive draws being to close to each other. Is there another purpose to this?

Answer 1

As it was already mentioned, this is called thinning, however, as Kruschke noticed it is "rarely useful" and quoting Link and Eaton (2012), he writes that

... basic conclusion of the article is that thinning of chains is not usually appropriate when the goal is precision of estimates from an MCMC sample. (Thinning can be useful for other reasons, such as memory or time constraints in post-chain processing, but those are very different motivations than precision of estimation of the posterior distribution.)

The paper published in Methods in Ecology and Evolution is freely avilable from the publisher.

So it is disputable if thinning is that much helpful, i.e. the inner loop may not be that important in the code and in many cases can be simply omitted with saving the whole simulation output.

Answer 2

Just to expand on what you and @Glen_b said: The point of the inner loop is to reduce correlation among sampled values. A common way to do this is by only taking every nth value, a process called Thinning.

From Wikipedia:

It is common to ignore some number of samples at the beginning and then consider only every th sample when averaging values to compute an expectation. … The reason for this is that (1) successive samples are not independent of each other but form a Markov chain with some amount of correlation; (2) the stationary distribution of the Markov chain is the desired joint distribution over the variables, but it may take a while for that stationary distribution to be reached.

Your example (the inner loop) does this slightly differently. Instead of reducing correlation between samples by taking every n^th value, it obscures it by randomizing each sample thin number of times.