# Within-chain and between-chain average difference in MCMC simulation

I have done 400 repetitions of a particular MCMC simulation (Metropolis–Hastings algorithm) to get a quantity of interest $N$. The simulation reaches its steady-state after ~$10^5$ iterations. The typical correlation length is around ~$10^5$ iterations too but to be on the safe side, I have used a burn-in time of $10^6$ iterations and I let the simulation run until I reach $10^7$ iterations.

I have calculated the average value for $N$ using two methods:

• By calculating the average value at the $10^{7\text{th}}$ between the 400 independent chains.
• And by calculating the average value within each chain (all values between iteration $10^6$ and iteration $10^7$)

Problem: I have performed a few checks (normality, t-tests, etc.) but the two values of $N$ are still significantly different (one is 8 times bigger).

Would someone know why this could be the case? I suppose this has to do with ergodicity, but I am clearly not sure of it because I cannot see why.

• By calculating the average value at the 10⁷th between the 400 independent chains, you are evaluating the mean of the Markov chain after 10⁷th iterations. Commented May 28, 2017 at 20:10
• That's true but still, as $10^7 >>10^5$ I would expect that "mean after $X$ iterations" to be equal to the time average within one chain.
– Will
Commented May 29, 2017 at 10:41
• We would need more details to understand the issue but 8 times larger is indeed a signal of (at least one) something wrong! Commented May 29, 2017 at 14:46