1
$\begingroup$

I have come across in a research paper that,

enter image description here

enter image description here

I do understand the logic. But the paper has't mentioned about the way of updating $\eta_{t}$. When I asked from the authors they said when we equate the first derivative to be zero, we could find how to update $\eta_{t}$.

But, I couldn't exactly figure out how to relate $\eta_{t}$ and $\eta_{t+1}$ from the first derivative.

Any insight about this is greatly appreciated.

Thanks.

Note:

enter image description here

and

enter image description here

$\endgroup$
2
  • 1
    $\begingroup$ Please give a reference to the paper. $\endgroup$
    – Tom Minka
    Commented Nov 5, 2014 at 9:09
  • $\begingroup$ hope you could get it from this link. link $\endgroup$
    – shani
    Commented Nov 5, 2014 at 9:28

1 Answer 1

2
$\begingroup$

Because of the nonlinear function $\pi(z_{t+1})$, you cannot directly solve for the joint mode. You would have to apply "iterative methods, e.g. Newton's method" as done for computing $\alpha$ earlier, with details described in (the non-existent) Appendix A.

I have a hard time believing that the methods in this paper are going to give good approximations. The reason is that when they solve for the joint mode they aren't using all of the information available about the variables. During the $\alpha$ computation they find the joint mode without using backward information. During the $\beta$ computation, they find the joint mode again but without using forward information. Only when computing posterior marginal do they compute the joint mode using all of the information. The approximation they are making is very local, so if the mode computed using $\alpha$ is far away from the mode computed using both, then $\alpha$ will be accurate in a location that you don't care about, and inaccurate in the location that you do care about.

$\endgroup$
6
  • $\begingroup$ This question might be stupid. What I thought is in usual variational bayesian inference we carry-out forward and backward message passing independently, without exchanging them. Is it wrong? $\endgroup$
    – shani
    Commented Nov 5, 2014 at 9:50
  • 2
    $\begingroup$ The exact $\alpha$ and $\beta$ messages do not depend on each other, but when you are making approximations you can (and should) make use of the information in the other direction. See Video lectures on Approximate Inference starting at slide 32, with more details in the paper Window-based expectation propagation for adaptive signal detection in flat-fading channels. $\endgroup$
    – Tom Minka
    Commented Nov 5, 2014 at 10:11
  • $\begingroup$ I had a look at the video. Could you please explain a bit more your last statement. Thanks alot :) $\endgroup$
    – shani
    Commented Nov 5, 2014 at 11:39
  • $\begingroup$ The question is answered in the first paragraph. You can just ignore the second paragraph if it makes no sense to you. $\endgroup$
    – Tom Minka
    Commented Nov 5, 2014 at 11:56
  • 1
    $\begingroup$ Ask the authors to send you Appendix A, since this information is supposed to be there. $\endgroup$
    – Tom Minka
    Commented Nov 5, 2014 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.