# Backward message passing in variational Bayesian inference

I have come across in a research paper that,

I do understand the logic. But the paper has't mentioned about the way of updating $\eta_{t}$. When I asked from the authors they said when we equate the first derivative to be zero, we could find how to update $\eta_{t}$.

But, I couldn't exactly figure out how to relate $\eta_{t}$ and $\eta_{t+1}$ from the first derivative.

Thanks.

Note:

and

• Please give a reference to the paper. Commented Nov 5, 2014 at 9:09
• hope you could get it from this link. link Commented Nov 5, 2014 at 9:28

Because of the nonlinear function $\pi(z_{t+1})$, you cannot directly solve for the joint mode. You would have to apply "iterative methods, e.g. Newton's method" as done for computing $\alpha$ earlier, with details described in (the non-existent) Appendix A.
I have a hard time believing that the methods in this paper are going to give good approximations. The reason is that when they solve for the joint mode they aren't using all of the information available about the variables. During the $\alpha$ computation they find the joint mode without using backward information. During the $\beta$ computation, they find the joint mode again but without using forward information. Only when computing posterior marginal do they compute the joint mode using all of the information. The approximation they are making is very local, so if the mode computed using $\alpha$ is far away from the mode computed using both, then $\alpha$ will be accurate in a location that you don't care about, and inaccurate in the location that you do care about.
• The exact $\alpha$ and $\beta$ messages do not depend on each other, but when you are making approximations you can (and should) make use of the information in the other direction. See Video lectures on Approximate Inference starting at slide 32, with more details in the paper Window-based expectation propagation for adaptive signal detection in flat-fading channels. Commented Nov 5, 2014 at 10:11