Just wondering what are the advantages of using one over the other. I'm just looking for some general answers here.
For starters:
$\begingroup$
VB gives a guaranteed lower bound for the likelihood.
EP is faster? VB has quite a lot of iterations till convergence, unless someone came up with a faster method.
With EP you have to specify the family of the approximate posterior
$\endgroup$
1
-
$\begingroup$ VB and EP minimize KL divergence from different directions. $\endgroup$– Ziqian XieCommented Oct 13, 2017 at 20:21
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
It depends a lot on the details of the problem being solved. You can find a tabular comparison between them here, which links to more information. You are right about VB generally requiring more iterations, however sometimes each iteration is much cheaper than an EP iteration. Your third bullet is worded like a disadvantage of EP but it's more like an advantage---you can use the same approximating family as VB if you want, but you don't have to.
-
10$\begingroup$ Ah, if it isn't the creator himself. Big fan of your work $\endgroup$ Commented May 28, 2014 at 1:17
-
$\begingroup$ Hi! The specified link is no longer functional, do you mind providing another way to access the referenced comparison? Thank you for all your work. $\endgroup$ Commented May 11, 2017 at 19:24
-
1
-
$\begingroup$ Apparently they moved the page again, there's the new link: dotnet.github.io/infer/userguide/… For the future, just search for "working with different inference algorithm" (include quotes). $\endgroup$– mugenCommented Sep 29, 2019 at 16:20
$\begingroup$
$\endgroup$
1
There are so many details that go into a coherent decision. In many cases, assuming you have an outcome in mind it may be helpful to implement both and the subsequently compare which result closest matches your desired outcome. As mentioned VB generally requires more iterations however it is often cheaper than other alternatives. Hope this helps. In closing I would recommend trying both to see which works. Post any issues you encounter and I hope to be more useful.
-
2$\begingroup$ Good points but it is not an answer. It would be more appropriate to make it a comment. $\endgroup$ Commented May 12, 2017 at 20:24