1
$\begingroup$

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP algorithm is equivalent to a stationary point of the following objective function

\begin{alignat}{2} &\!\min \limits_{\boldsymbol{\lambda}} \max \limits_{{\boldsymbol{\lambda}}_{\backslash k}} &\qquad& (K-1)\log \int p(\theta) \exp({\bf{s}}^\top \boldsymbol{\lambda}) d\theta - \sum_{k=1}^K \log \int p(\theta) p(y_k\mid \theta) \exp({\bf{s}}^\top \boldsymbol{\lambda})d\theta\\ &\text{s.t.} & & (K-1){\boldsymbol{\lambda}} = \sum_{k=1}^K \boldsymbol{\lambda}_{\backslash k} \end{alignat}

However they don't show how it's derived. I looked everywhere and I couldn't find an answer. I tried to do the proof myself but failed. I think the proof involves two steps: showing that stationary points of teh objective function above are also fixed points of the EP algorithm, and that viceversa fixed points of the EP algorithm are stationary points of this objective function. Can someone help?

$\endgroup$
2
$\begingroup$

The paragraph under (14) in the original paper explains how to show that the EP fixed points are in one-to-one correspondence with stationary points of the objective. A similar derivation is done in Power EP.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.