1
$\begingroup$

I am estimating a model using MCMC (Gibbs Sampling). Because of the complexity of the model, I have been running two chains with many iterations.

A plot of the draws for each parameter reveals a spike in the variance of the draws. After an additional few thousand iterations, the plot returns to the region of the distribution where has spent most of its time. I have run two chains in parallel with different starting values and different seeds for the random number generator. One chain does this at around 50,000th iteration, and the other does this around 25,000th iteration.

Here is a plot of a parameter sampled from an inverse gamma posterior in the first chain:

Posterior Sample from Inv. Gamma Distribution

Here are similar results - but appropriately more symmetric - for a different parameter sampled from a normal posterior from the same chain:

Posterior Sample from Normal Distribution

The entire support of these distributions will be covered over an infinite number of iterations, but these spikes seem odd for a finite number of samples.

I have looked through my code to determine what could be going wrong and not found any apparent issues. I have seen a similar question asked here, but there hasn't been much discussion.

Any feedback on this behavior would be appreciated!

Improve this question
$\endgroup$
7
  • $\begingroup$ What do you mean "inverse gamma posterior" and "normal posterior"? Do you mean "inverse gamma full conditional" instead? Also, it would be good to share some code here, because there is not enough information about the model or the sampler to say anything with certainty. $\endgroup$
    – Greenparker
    Commented Mar 2, 2018 at 15:31
  • $\begingroup$ What are your priors on the variable in question? $\endgroup$
    – jbowman
    Commented Mar 2, 2018 at 15:37
  • $\begingroup$ Yes, @Greenparker, I do mean the full conditional. I would like to share some code here, but there's too much to share. $\endgroup$
    – Alex
    Commented Mar 2, 2018 at 15:48
  • $\begingroup$ @jbowman, this is a regression model with the noninformative prior proportional to sigma^-2, where sigma is the standard deviation of the residuals. The likelihood is normal. I will update the original post. $\endgroup$
    – Alex
    Commented Mar 2, 2018 at 15:50
  • $\begingroup$ You might want to use a more-informative weakly informative prior to prevent this sort of thing from happening; see the answers to this question: stats.stackexchange.com/questions/6493/… and the excellent Gelman paper pointed to at the link. $\endgroup$
    – jbowman
    Commented Mar 2, 2018 at 15:53

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.