10
$\begingroup$

Just wondering what are the advantages of using one over the other. I'm just looking for some general answers here.

For starters:

  • VB gives a guaranteed lower bound for the likelihood.
  • EP is faster? VB has quite a lot of iterations till convergence, unless someone came up with a faster method.
  • With EP you have to specify the family of the approximate posterior
  • $\endgroup$
    1
    • $\begingroup$ VB and EP minimize KL divergence from different directions. $\endgroup$ – Ziqian Xie Oct 13 '17 at 20:21
    13
    $\begingroup$

    It depends a lot on the details of the problem being solved. You can find a tabular comparison between them here, which links to more information. You are right about VB generally requiring more iterations, however sometimes each iteration is much cheaper than an EP iteration. Your third bullet is worded like a disadvantage of EP but it's more like an advantage---you can use the same approximating family as VB if you want, but you don't have to.

    $\endgroup$
    4
    • 9
      $\begingroup$ Ah, if it isn't the creator himself. Big fan of your work $\endgroup$ – sachinruk May 28 '14 at 1:17
    • $\begingroup$ Hi! The specified link is no longer functional, do you mind providing another way to access the referenced comparison? Thank you for all your work. $\endgroup$ – Mircea Mironenco May 11 '17 at 19:24
    • 1
      $\begingroup$ I've fixed the link. $\endgroup$ – Tom Minka May 12 '17 at 20:09
    • $\begingroup$ Apparently they moved the page again, there's the new link: dotnet.github.io/infer/userguide/… For the future, just search for "working with different inference algorithm" (include quotes). $\endgroup$ – mugen Sep 29 '19 at 16:20
    0
    $\begingroup$

    There are so many details that go into a coherent decision. In many cases, assuming you have an outcome in mind it may be helpful to implement both and the subsequently compare which result closest matches your desired outcome. As mentioned VB generally requires more iterations however it is often cheaper than other alternatives. Hope this helps. In closing I would recommend trying both to see which works. Post any issues you encounter and I hope to be more useful.

    $\endgroup$
    1
    • 1
      $\begingroup$ Good points but it is not an answer. It would be more appropriate to make it a comment. $\endgroup$ – Michael R. Chernick May 12 '17 at 20:24

    Your Answer

    By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

    Not the answer you're looking for? Browse other questions tagged or ask your own question.