I have two questions in Energy function of expectation propagation. 1. Seeing the video of [1] (slide 24 of the second part), Minka says that, the evidence is the following: $$ Z = \left( \int q(\mathbf{x}) d\mathbf{x} \right) ^{1-n} \prod_{i=1}^{n} \int \frac{f_i (\mathbf{x})}{\tilde{f}_i(\mathbf{x}) } q(\mathbf{x}) d\mathbf{x} \quad \text{ (Equation a)} $$ But in [2] he introduces the primal energy function as it is shown in equation (6) of [2]. $$ \sum_i \int_{\mathbf{x}} \hat{p}_i(\mathbf{x}) \log \frac{ \hat{p}_i(\mathbf{y}) }{ f_i(\mathbf{x}) p(\mathbf{x}) } d\mathbf{x} - (n-1) \int_\mathbf{x} q(\mathbf{x}) \log \frac{q(\mathbf{x})}{p(\mathbf{x})} d\mathbf{x} \quad \text{ (Equation b)} $$ with $$ \hat{p}_i = \frac{1}{Z} q^{\setminus i} f_i(\mathbf{x}) $$ As I understand, we are trying to minimze $-\log Z$ which must be the same as equation (6) of [2]: $$ \log Z = \sum_{i=1}^{n} \log \int \frac{f_i (\mathbf{x})}{\tilde{f}_i(\mathbf{x}) } q(\mathbf{x}) d\mathbf{x} - (n-1) \log \int q(\mathbf{x}) d\mathbf{x} \quad \text{ (Equation c)} $$ However, I don't see how these are the same (UPDATE: I mean (Equation b) and (Equation c) ). Any idea on this?
- At [3] after equation 64 it says that : "The scale $s_a$ that minimizes local $\alpha$-divergence is ... ". Let's say we are in the case of KL divergence, and want to find $s_a$ for $\alpha=1$ (correct?). Since $s_a$ is defined to normalize the distribution $\tilde{f}_a(\mathbf{x})$, thus: $$ s_a = \frac{1} {\int \tilde{f}_a(\mathbf{x}) } $$ (right?) and $$ \tilde{f}_a(\mathbf{x})^{new} = \frac{ \arg\min_{q \in \mathcal{Q}} KL \left( q(\mathbf{x}) \frac{f_a(\mathbf{x})}{ \tilde{f}_a(\mathbf{x}) } || q^{\prime} \right) }{ q(\mathbf{x}) \frac{1}{ \tilde{f}_a(\mathbf{x}) } } \quad \text{ (Equation d)} $$ The paper [3] says that (for $\alpha=1$): $$ s_a = \frac{ \int q(\mathbf{x}) \frac{f_a(\mathbf{x})}{ \tilde{f}_a(\mathbf{x}) } d\mathbf{x} }{ \int q(\mathbf{x}) d\mathbf{x} } \quad \text{ (Equation e)} $$ However I don't see why. Any hints?
[1] videolectures.net/mlss09uk_minka_ai/
[2] http://research.microsoft.com/en-us/um/people/minka/papers/ep/minka-ep-energy.pdf
[3] http://www.seas.harvard.edu/courses/cs281/papers/minka-divergence.pdf
