2
$\begingroup$

I wonder if only me feel there are certain connections between them, I googled it for a long time, but found no where mentioned these two method. But to me, they indeed looked so related, Could anyone help?

$\endgroup$
4
$\begingroup$

The purpose of coordinate ascent is to maximize a function, whereas the purpose of Gibbs sampling is to draw samples from a probability distribution. The two methods are loosely related in the sense that they both fix all coordinates except for the one being updated, and take turns updating the individual coordinates. They differ in that coordinate ascent always steps uphill, whereas Gibbs sampling draws a sample from the marginal distribution. The probability density associated with the new point may be higher or lower than the previous point. If you were to apply coordinate ascent in place of Gibbs sampling, you'd converge to a local mode of the distribution, as in maximum likelihood or MAP estimation (as long as coordinate ascent doesn't get stuck). The sequence of points would no longer be a valid sample from the distribution. It doesn't make sense to talk about applying Gibbs sampling in place of coordinate ascent, because coordinate ascent deals with functions that aren't necessarily probability distributions.

You might also be interested to think about the similarities/differences between Metropolis-Hastings and simulated annealing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.