Why do we only look at the sample from MCMC? (And ignore the sample points' posterior probabilities?) [duplicate]

Question

I'm new to MCMC, so this might be obvious.

Let's say we're using MCMC to estimate a posterior distribution. We run MCMC, and it returns a representative sample from that posterior distribution.

The sample is just a collection of data points. To actually use it, we just plot a (normalized) histogram of those data points to draw our estimated posterior.

However, we also know the posterior probability of each data point in the sample! This is even explicitly calculated in the Metropolis MCMC algorithm. Why do we ignore this?

---

Edit: Here's a similar question.

Answer 1

You are right, if we knew the posterior probabilities, we wouldn't need the samples, since we would know the posterior distribution, and the whole point of MCMC is to learn the posterior distribution. We use MCMC to draw samples from the distributions in cases where we don't know the probabilities, or more precisely, we know them up to a constant. Bayes theorem is

$$ p(\theta|X) = \frac{p(X|\theta)\,p(\theta)}{\int \, p(X|\theta)\,p(\theta) \, d\theta} $$

the problem is that computing the normalizing constant $\int \, p(X|\theta)\,p(\theta) \, d\theta$ in many cases is a hard computational problem. Hopefully, to find maximum of such distribution (maximum a posteriori estimation), or to sample from it, we just need unnormalized density

$$ p(\theta|X) \propto p(X|\theta)\,p(\theta) $$

i.e. you just need to multiply likelihood by prior and do not need to take the integral.