# How to sample using MCMC from a posterior distribution in general?

Assume one has the posterior distribution of a parameter, $p(\theta|y)$ and what I mean by having it is that for each point of $\theta$, one can use Monte Carlo method+MCMC to calculate the $p(\theta|y)$. Now my question is if I want to sample from $p(\theta|y)$, them basically I have to do one Gibbs sampling(for example) to sample from distribution and at any point I have to run Monte Carlo method on the point to calculate $p(\theta|y)$'s value right? i.e. it needs two loops, one inside of the other. Is this correct?

As I got an answer to this question and I thought maybe my question is vague I will try to clarify it a bit more:

From what I know by reading for a week the whole time about Monte Carlo method and MCMC, I understood(correct me if I am wrong) that: $$p(\theta|y)=\frac{p(y|\theta)p(\theta)}{\int_{\Theta}{p(y|\theta)p(\theta)}\text{d}\theta}.$$

Now if you consider that we only have a sampling algorithm for $\theta$ and we can only calculate $p(y|\theta)$ explicitly(and not the other functions!), therefore to get values from $p(\theta|y)$ one needs to numerically integrate the denominator. And for each value of this posterior one needs to apply a sampling scheme like Gibbs sampling to generate a sample of $p(\theta|y)$; each new transition in the parameter space should then sample from the distribution which is $p(\theta|y)$ here and to calculate that the above proportion should be computed.

## 2 Answers

We don't use MCMC to calculate the $p(\theta | y)$ for each value (or many values) of $\theta$. What MCMC (or the special case of Gibbs sampling) does is generate a (large) random sample from $p(\theta | y)$. Note that $p(\theta | y)$ is not being calculated; you have to do something with that vector (or matrix) of random numbers to estimate $p(\theta)$. Since you're not calculating $p(\theta)$ for lots of values of $\theta$, you don't need a Gibbs (or MCMC) loop inside a $\theta$ loop - just one (long) Gibbs (or MCMC) loop.

EDIT in response to an update to the question: We do not need to integrate the distribution to get the constant of integration (CoI)! The whole value of MCMC is is found in situations where we can't calculate the CoI. Using MCMC, we can still generate random numbers from the distribution. If we could calculate the CoI, we could just calculate the probabilities directly, without the need to resort to simulation.

Once again, we are NOT calculating $p(\theta|y)$ using MCMC, we are generating random numbers from $p(\theta|y)$ using MCMC. A very different thing.

Here's an example from a simple case: the posterior distribution for the scale parameter from an Exponential distribution with a uniform prior. The data is in x, and we generate N <- 10000 samples from the posterior distribution. Observe that we are only calculating $p(x|\theta)$ in the program.

x <- rexp(100)

N <- 10000
theta <- rep(0,N)
theta <- cur_theta <- 1  # Starting value
for (i in 1:N) {
prop_theta <- runif(1,0,5)  # "Independence" sampler
alpha <- exp(sum(dexp(x,prop_theta,log=TRUE)) - sum(dexp(x,cur_theta,log=TRUE)))
if (runif(1) < alpha) cur_theta <- prop_theta
theta[i] <- cur_theta
}

hist(theta)


And the histogram: Note that the logic is simplified by our choice of sampler (the prop_theta line), as a couple of other terms in the next line (alpha <- ...) cancel out, so don't need to be calculated at all. It's also simplified by our choice of a uniform prior. Obviously we can improve this code a lot, but this is for expository rather than functional purposes.

Here's a link to a question with several answers giving sources for learning more about MCMC.

• Thank you very much, but I think my question was vague maybe, and therefore I updated it. – Cupitor Oct 24 '13 at 2:22
• I've updated my answer in response to your question updates. – jbowman Oct 24 '13 at 17:09
• I think there is a misunderstanding(probably from my side) here again! :) Even if you can calculate $p(\theta|y)$ pointwise doesn't mean that you can sample from it. THAT! is exactly when MCMC sampling comes into play, right? – Cupitor Oct 24 '13 at 17:36
• It's OK, it happens like that sometimes. I think you are still thinking in terms of calculating probabilities; we don't do that with MCMC, we generate random numbers - and we don't need to calculate probabilities as part of the process. It's a clever way to AVOID having to calculate probabilities, which, due to our (problem-specific) inability to calculate the constant of integration, we can't do anyway. – jbowman Oct 24 '13 at 17:40
• To expand on my comment - with "typical" random number generation, you DO have to be able to calculate a probability or a cum. density (not always, there are schemes which avoid that.) So your path to the random number stream leads through the probability distribution. With MCMC, you don't have to calculate a probability or a cum. density, just an unnormalized probability (likelihood ratio * prior). So your path to the random number stream does not lead through the probability distribution. If it did, you'd need that integral, but it doesn't. – jbowman Oct 24 '13 at 17:44

MCMC is a family of sampling methods (Gibbs, MH, etc.). The point of MCMC is that you cannot sample directly from the posterior distribution that you mentioned. The way MCMC works is a Markov Chain (the first MC in MCMC) is identified whose stationary distribution is the posterior that you are interested in. You can sample from this Markov Chain and when it converges to its equilibrium distribution, you are essentially sampling from the posterior distribution that you are interested in.

• I know that I cannot sample from it directly but the point about the denominator of the above relation is that it needs to be calculated using numerical integration, and that apparently needs a Monte Carlo method integration, namely generating a sum over different values of $\theta$ sampled from the prior! – Cupitor Oct 24 '13 at 16:14
• I'm not sure what you are saying. MCMC does not calculate the denominator, it generates samples from a Markov Chain. Often, the main purpose of using MCMC is because that integral is intractable. So MCMC is a way of getting samples from the posterior without having to solve that integral. So the denominator does not "need" to be calculated. – user1893354 Oct 24 '13 at 17:05