2
$\begingroup$

I've read a number of times that autocorrelation is highly undesirable in any MCMC samplers and, when it is high, your number of effective samples will be reduced, which in turn means that the approximation of the target distribution suffers. I've largely taken this for granted and accepted it without much disbelief.

My best guess is that ~"If past samples are strong predictors of future samples then the ability for the sampler to explore all regions of the target distribution cannot be taken for granted." For example, if sampling from a 1D bimodal distribution, a Metropolis sampler would need to pass through the "valley" between peaks to reach the unexplored peak (of course depending on perturbation size and proposal distribution.)

Could someone explain in more explicit terms why autocorrelation is so bad for MCMC sampler performance?

$\endgroup$

1 Answer 1

6
$\begingroup$

Isn't it just like sampling in general? You ideally want to sample IID from the target distribution, so that sample mean looks like population mean. If you don't sample IID, you have a sampling bias and you have to correct for that, which costs you samples in the sense that only a subset of your samples are independent (because autocorrelation decays).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.