MCMC: sampling or optimization?

Question

I need a conceptual clarification on Markov Chain Monte Carlo (MCMC). I have read that MCMC is used to sample from a posterior distribution when the shape of the likelihood distribution $p(x | \theta)$ can be roughly estimated.
However, the applications I have seen so far are used to find the parameters that better describe this likelihood distribution by an iterative stochastic process. According to this, we iteratively propose parameters that are accepted if a specific requirement is met, e.g. the ratio between the unnormalized posterior distribution given the proposed parameters $p(\theta' | x)$ and the posterior given the current parameter $p(\theta' | x)$ exceeds a certain threshold (this is the Metropolis algorithm). If what I am saying is correct, we are not trying to sample from the posterior, but we are trying to optimize the underlying distribution that could have generated the data.
What am I missing?

Answer 1

You are referring to the Metropolis algorithm. What it does, at each iteration it proposes a new value $x'$ that is accepted with probability $\min(p(x') / p(x), 1)$ what enables us to sample from the distribution $p$. This may look similar to simulated annealing, an optimization algorithm, but they are not the same. First, Metropolis generates a sample at each step, simulated annealing just cares about the current value. After using the Metropolis algorithm, you are left with multiple samples from the distribution $p$. You can use those samples to find a region of the distribution with the highest concentration of the samples, the mode of the distribution (the maximum), but you can also use them for other purposes. Moreover, the acceptance criteria used in simulated annealing leads to exploring the possible values first, but as temperature $T$ decreases, it focuses on exploiting the most promising values and ultimately finding the optimal one. The Metropolis algorithm does only the exploration, so if you care only about finding the mode of the distribution, it wouldn't be the most efficient way to do it.