# MCMC Intuition for Expected Value Estimate

I believe I have a fundamental misunderstanding of MCMC. We seek to estimate the expected value of a target distribution using its integral form. We consider a markov chain of samples from a simpler distribution, which, after a burn-in phase, gets arbitrarily close to the stationary distribution, designed carefully to be the target distribution, as $$n \rightarrow \infty$$.

It is computationally inefficient to evaluate the target density in regions of the parameter space which have little contribution to the desired expectation. So, we seek to find the "typical set" (high density, high volume regions of parameter space) and take many samples from that region.

What I don't understand is that, if we are designing our samples to take disproportionately many samples from the typical set, won't we be overestimating our expected value? My intuition is that this would be similar to sampling from a $$Unif[0,1]$$ but the vast majority of samples being around $$[0.7, 0.9]$$, which would skew our expected value estimate upward using our sample mean. What am I misunderstanding here?

To convince yourself whether the MCMC yields correct sampling distribution, perhaps you can do the following little simulation: for different values of $$n$$ (say, $$10^3, 10^4, .... 10^7$$) obtain $$K (=1000)$$ samples from the target distribution and observe how the mean value changes with $$n$$.