I am trying to understand the difference between:

Wikipedia says:

Expectation Propagation differs from other Bayesian approximation approaches such as Variational Bayesian methods.

Why isn't EP considered a Variational Bayes method? Isn't EP Bayesian, and relies on message-passing to approximate the posterior? What makes a method Variational Bayes?

Also, what about the following methods. Can they be considered Variational Bayes?

  • Sum-product
  • Mean-field methods
  • Bethe-Kikuchi approximations

To me VB just means we will approximate posterior by turning the problem into a minimization problem and then changing the solution space. In that sense, EP is a type of VB algorithm, because it does exactly that. When I think of VB, though, the first thing I think of is mean-field, and if I had to guess I would think that whoever wrote that EP is "different" from VB might have had mean-field algorithms in mind.

This source, which I take to be canonical, explicitly classifies all algorithms you mention as VB (i.e. Bethe-Kikuchi, sum-product, EP, and mean-field).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.