Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am learning MCMC in class, and I encounter one question about the relationship between posterior probability and likelihood function.
In our lecture, the professor asked us to take samples from the posterior distribution, however, he used the likelihood function as the target distribution $P(X)$ in the Metropolis-Hastings algorithm. So I am confused why the sample generated by the likelihood using MCMC can be seen as a sample from the posterior distribution?
The probability density function (PDF) for the posterior distribution is proportional to the product of the likelihood and the prior distribution: $$ p(\theta|y) \propto p(y|\theta)\,p(\theta), $$ where $y$ denotes the observed data and $\theta$ denotes the unobserved parameter. If the prior is flat, $$ p(\theta) \propto 1, $$ and the likelihood is normalizable, $$ \int p(y|\theta)\,d\theta < \infty, $$ then the posterior is proportional to the likelihood: $$ p(\theta|y) \propto p(y|\theta). $$ The Metropolis-Hastings algorithm can be applied to the likelihood in this case to produce draws from the posterior.
