4
$\begingroup$

I know that "conjugate prior" is to help us calculate the the denominator of the Bayes formula(to make the calculations easier). And I just learnt to approximate the inference by mean field approximation to help us calculate the denominator of the Bayes formula(make the calculations easier).

What is the relation between the two? Why do we need "mean field approximation" If we have a conjugate prior?

$\endgroup$
4
$\begingroup$

Priors are not chosen for convenience but for reflecting one's own beliefs or absence thereof about the parameters of the model. There is thus no foundational reason for always choosing conjugate priors, which main justification is computational. Furthermore,

  1. Conjugate priors only exist for exponential family models. Outside these models, there is no easily handled prior and computation tools are always needed.
  2. Variational Bayes approximations are themselves based on exponential family approximations to the exact model.
$\endgroup$
4
$\begingroup$

There's no relation whatsoever. If you use conjugate priors, then the solution exists in closed-form, so you don't need to use things like MCMC or approximate inference to obtain the solution.

$\endgroup$
  • $\begingroup$ So you mean that if we have conjugate priors, we don't have to approximate inference using "mean field"? So what is the point of "approximate inference" if we can always use conjugate priors as an assumption? $\endgroup$ – floyd Aug 18 at 7:56
  • 2
    $\begingroup$ @floyd we do not "always" use conjugate priors. For some problems we can use them, but for vast majority of cases we cannot. $\endgroup$ – Tim Aug 18 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.