2
$\begingroup$

My model contains five parameters. I want to make Bayesian estimation, but the Bayes estimates can not be obtained in closed form. So, I used Metropolis-Hastings to generate MCMC samples from conditional posterior density of each parameter. The trace and Auto-correlation plots were used to evaluate the generated sample. The trace plots for four parameters are random and the Auto-correlation plots are decreasing whereas for the fifth parameter(I will referred as alpha1), the trace plot is not random and the lags in Auto-correlation plot is not decreasing.

Discussion about alpha1

The acceptance rate is 0.99905 (It is too high) and the density plot is multimodal like the following graph

enter image description here

Trace plot enter image description here

Auto-correlation plot enter image description here

when I change the variance of the proposal density (absolute normal density) from 0.0005 to 0.5 the Acceptance rate becomes 0.1515 and the trace plot is horizontal line and the density function becomes uni-modal enter image description here

I read MCMC for multimodal posterior and this chapter from this book Evaluating Markov Chain Monte Carlo (MCMC) Algorithms. I am beginner in MCMC, I would be appreciated if one can help me to detect the problem. Is it multi modal or the correlation between parameters are high.

Many thanks in advance. Edit mixture 5 Normal density and trace plots enter image description here mixture 3 Normal trace plots enter image description here

mixture 3 Normal density plot enter image description here

| cite | improve this question | |
$\endgroup$
1
$\begingroup$

Multimodality is often a cause of poor convergence or lack thereof for MCMC algorithms. Without any detail available about the posterior you aim at approximating, it is hard to tell whether or not you are facing such a case, because the multimodality on the first (density) figure is most likely an artefact reflecting the very slow move of the (sub) Markov chain in the $\alpha$ space.

What is more worrying is the trace plot you produce for $\alpha$ when the scale of the proposal density moves to $0.5$ as it seems the sequence does not change at all. The density you obtain is again an artefact I believe, namely a single Normal density produced by the kernel density estimator of R.

I would suggest using a sequence of scales in the Metropolis proposal, from the lowest 5e-5 to the largest 5e-1 (or higher), chosen at random at each iteration, meaning the use of a mixture proposal in the Metropolis acceptance ration. This means using the sum of the five Normal proposals in this ratio, rather than the Normal with the scale that has been chosen for that iteration.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Many thanks, I tried to use mixture of Gaussian proposal using the sum of the five Normal proposals which is better than my suggestion. when I use mixture Gaussian using the sum of the three Normal proposals it better than based on five Normal proposals. The trace plot better than before and partly solving multi-modal. I edit the question to present the results $\endgroup$ – userr Nov 1 '17 at 19:46
  • $\begingroup$ Your traceplot does not exhibit convergence or stationarity. You should attempt ten times or 100 times more iterations. $\endgroup$ – Xi'an Nov 1 '17 at 20:11
  • $\begingroup$ Do you know how can I make thinning in Metropolis-Hastings algorithm in R. I wrote the code not a function in R. $\endgroup$ – userr Nov 6 '17 at 7:47
  • $\begingroup$ Thinning means taking one simulation out of 10, 100, 1000. $\endgroup$ – Xi'an Nov 6 '17 at 9:22
  • $\begingroup$ I want to apply this in R , I tried to make thinning after getting the mcmc samples by using 'a<-seq(1,10000,by=100) alpha1.adj<-alpha1[a]' $\endgroup$ – userr Nov 6 '17 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.