I need a conceptual clarification on Markov Chain Monte Carlo (MCMC). I have read that MCMC is used to sample from a posterior distribution when the shape of the likelihood distribution $p(x | \theta)$ can be roughly estimated.
However, the applications I have seen so far are used to find the parameters that better describe this likelihood distribution by an iterative stochastic process. According to this, we iteratively propose parameters that are accepted if a specific requirement is met, e.g. the ratio between the unnormalized posterior distribution given the proposed parameters $p(\theta' | x)$ and the posterior given the current parameter $p(\theta' | x)$ exceeds a certain threshold (this is the Metropolis algorithm). If what I am saying is correct, we are not trying to sample from the posterior, but we are trying to optimize the underlying distribution that could have generated the data.
What am I missing?


1 Answer 1


You are referring to the Metropolis algorithm. What it does, at each iteration it proposes a new value $x'$ that is accepted with probability $\min(p(x') / p(x), 1)$ what enables us to sample from the distribution $p$. This may look similar to simulated annealing, an optimization algorithm, but they are not the same. First, Metropolis generates a sample at each step, simulated annealing just cares about the current value. After using the Metropolis algorithm, you are left with multiple samples from the distribution $p$. You can use those samples to find a region of the distribution with the highest concentration of the samples, the mode of the distribution (the maximum), but you can also use them for other purposes. Moreover, the acceptance criteria used in simulated annealing leads to exploring the possible values first, but as temperature $T$ decreases, it focuses on exploiting the most promising values and ultimately finding the optimal one. The Metropolis algorithm does only the exploration, so if you care only about finding the mode of the distribution, it wouldn't be the most efficient way to do it.

  • $\begingroup$ My confusion is related to your first sentence. Based on what I have seen, the proposed values are the distribution parameters (e.g. mean and std dev. for a Gaussian likelihood), not the data points. That is, we collect multiple values for the mean of the distribution. Isn't this trying to optimize the mean then? And if so, why don't we just store the x: argmax p(x)? $\endgroup$
    – maurock
    Commented May 21, 2021 at 7:12
  • $\begingroup$ @maurock we are collecting values from the distribution, not from its mean. So there’s nothing “optimal” about them. $\endgroup$
    – Tim
    Commented May 21, 2021 at 8:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.