I am experimenting with MCMC methods and have implemented a basic Metropolis-Hastings algorithm.
One potential issue with this is that MH posterior samples are autocorrelated. I could verify that mine were as well.
This is often referred to as a known issue and various fixes are suggested such as "thinning" where only every N:th sample is used.
My basic question is: why (or rather when) is this a problem?
In my problem, I just want to get a good posterior distribution of a model parameter, e.g. density and moments.
Even if the samples are autocorrelated, if I have enough of them, shouldn't I be fine?
Let's say that I am not fine (as I am sure someone will explain):
What if I run my chain until it converges, then grab 5000 samples and draw from them in a randomized fashion. Clearly the samples will then not be autocorrelated. Isn't this a better solution than "thinning"?
In my experience when I used thinning I had to basically throw away 20 times the samples I actually used, causing excessive execution time.
Any feedback is welcome.