3
$\begingroup$

I am reading this tutorial on variational inference and wonder why the statement in the question title which is mentioned on page 3 is true.

$\endgroup$

2 Answers 2

4
$\begingroup$

Because in Bayesian analyses, you start off with the likelihood and priors, which directly yields the unnormalized joint distribution. So let's say $\Theta$ is the set of all parameters. Since the likelihood is not always equal to the density but proportional, it is clear that $$L\left(\Theta|y\right)p\left(\Theta\right) \propto p\left(y,\Theta\right).$$ But oftentimes integrating out $\Theta$ will be hard. This is why it's better to work with $p\left(y,\Theta\right)$ than $p\left(\Theta|y\right)$.

In short, sometimes you won't be able to do the integration.

$\endgroup$
2
  • 1
    $\begingroup$ Then what is the need for emphasising on the "exponential family" part? $\endgroup$ Jul 20, 2016 at 6:37
  • 2
    $\begingroup$ Because you need to take the logarithm and then take the expectation in variational inference. If it's exponential family, algebra becomes much easier. $\endgroup$ Jul 20, 2016 at 6:40
1
$\begingroup$

To compute P(x|D) you typically need P(D), which requires integrating x out.

$\endgroup$
1
  • 2
    $\begingroup$ Then what is the need for emphasising on the "exponential family" part? $\endgroup$ Jul 20, 2016 at 6:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.