Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have this confusion related to Monte Carlo Markov Chain method. I know that Monte Carlo method is used to get the sample mean instead of calculating the high dimensional integration which is not tractable.
However, I didn't get what's the role of Markov chain in this. I mean if we take samples from the posterior distribution of the parameters, then we can average the function of the parameters to get the expectation or mean. I didn't get what the role of Markov chain is here?
In Standard Monte Carlo integration, as you correctly state, you draw samples from a distribution and approximate some expectation using the sample average rather than calculating a difficult or intractable integral. So you are exploiting the Strong Law of Large Numbers to do this: \begin{equation} \mathbb{E}_{\pi} [t(\boldsymbol{\theta})] = \int t(\boldsymbol{\theta}) \pi(\boldsymbol{\theta})d\boldsymbol{\theta} \approx \sum_{i=1}^n t(\boldsymbol{\theta}_i) \end{equation} where each $\boldsymbol{\theta}_i \sim \pi$, provided $n$ is suitably large.
The problem is that in some cases you can't actually draw samples from $\pi$ directly - if you only know $c\pi$ for example, where $c$ is some unknown constant. This often happens in Bayesian Inference problems, where we have $posterior \propto likelihood \times prior$. Here one approach is to use some sort of re-sampling method, whereby we draw samples from some distribution $q$, and then adjust them in some way to make inferences about $\pi$. In rejection sampling, for example, some samples from $q$ are rejected, so that those remaining resemble a data set that would be likely under $\pi$. In importance sampling, the samples from $q$ are weighted based on their importance in inferring information about $\pi$.
Markov Chain Monte Carlo (MCMC) is another re-sampling technique, but this time samples are dependent on each other in some way. This can help or hinder inference, but crucially in high-dimensional problems MCMC methods can often be much more efficient than rejection sampling or importance sampling, so have become popular here.
This website may be of interest:
http://www.lancs.ac.uk/~jamest/Group/stats3.html (An informal guide to Monte Carlo methods written by some PhD students at Lancaster University)
A couple of good text books are:
Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference - Gamerman & Lopes (2006)
Markov Chain Monte Carlo in Practice - Gilks, Richardson & Spiegelhalter (1995).
First of all, Monte Carlo methods are not intended just for estimation of integrals. It is intended to sample random variables from distribution and the "by product" is aproximation of integrals. The reason behind the Markov chains is that MCMC methods/algorithms, e.g. Metropolis-Hastings produce a Markov Chain which has stationary distribution exactly the same from which you desire to obtain random draws. And the beauty of MCMC is that you can have your distribution just up to a normalizing constant. And the desire to obtain random samples from your distribution is not confined just to obtain estimates of expectation or variance.
