Expectation Maximisation vs Expectation Propagation in the context of Bayesian Networks

Question

I am confused about Expectation Maximisation and Expectation Propagation algorithms in the context of Bayesian Networks, especially whether one comprise another.

• What is the difference between expectation maximisation and expectation propagation?
• Is expectation propagation special case of expectation maximisation in the sense that it is used in Bayesian Networks to update parameters?

For example, in Baum-Welch algorithm, an EM algorithm, can I use EP to approximate the posterior? Or is it used only in the context of message passing like TrueSkill model (is 'propagation' part derives from that?).

Answer 1

Expectation maximisation is an optimisation algorithm for finding the maximum likelihood estimate of certain variables (let's call them "parameters"), while integrating out other variables (let's call them "latent"). It returns one value of the parameters. Expectation propagation is an algorithm for approximating the posterior distribution of all random variables (parameter and latent) in a Bayesian network.

Baum-Welch gives you a point estimate for the parameters of a Hidden Markov Model. EP can give you an approximate posterior over the parameters. EP is not a special case of EM. You could think of EM as a special case of EP, where you deliberately throw away uncertainty about the parameters at each step.