I am interested in generating samples from a density $\pi(\theta)$ to construct a histogram for $\pi(\theta)$ and to use these samples to generate samples of $f(\theta)$ for some function $f$. I may be interested in probabilities of $f(\theta)$ but my main goal is not the computation of statistics of $f(\theta)$ like its mean or variance. I am more interested in constructing histograms of $f(\theta)$ and kernel densities.

Since Metropolis MCMC returns correlated samples, I am curious as to how I should employ it for my objective. According to this post, the ergodicity of the chain enables the use of the samples for constructing histograms.

I then wonder about the practical side of MCMC sampling: After reading some books, it appears to be advocated that the step size of the chain be chosen correspondingly such that the correlation between consecutive samples is as small as possible. It seems that the objective of these practitioners is to compute the mean, variance of $\theta$. It makes sense because if the samples are almost independent, then the performance is comparable to standard Monte Carlo.

But if your goal is to construct the histogram, what benefits would you have from finding the optimal step size such that consecutive samples are highly uncorrelated? is it for better exploration of the state space?

I am asking this question because my trace plot (for the 2 most dominant components of a 10-dimensional random vector) look like:

enter image description here

This already has an acceptance rate of 23%. I've played around with other step sizes but I couldn't get a beautiful white noise trace plot as in the figures here.

Comments appreciated. Thanks!


If you are interested in constructing a histogram then there is no reason you need the samples from your MCMC to be (close to) uncorrelated. It is legitimate to take a long string of correlated values, discard the burn-in, and form a histogram with all the remaining values. Having said this, please bear in mind that a histogram is just a poor-man's kernel density plot; you are better off taking your MCMC values and making a kernel density plot with an appropriate kernel.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.