I am reading the paper titled "A Variational Bayesian Framework for Graphical Models" by Hagai Attias ( http://www.gatsby.ucl.ac.uk/publications/papers/03-2000.pdf ). I do not follow how Hagai got equation (2). I realize that one can write $P(Y|m)$ as $\int P(Y, \theta) d(\theta)$. Then taking log on both sides and dividing and multiplying by $q(X) q(\theta)$, by applying Jensens inequality one gets $\int d(\theta) q(X)q(\theta) log(\frac{P(Y,\theta)}{q(X)Q(\theta)})$ on the LHS. What throws me off is Hagai also has X in there. So the RHS is really $P(Y,X|m)$ isnt it, because the $X$ has not been marginalized in the integral? Also not sure what is the difference between $X$ and $\theta$ is in this paper. The author says $\theta$ are simply additional hidden nodes, what is the difference then and what is the need to have X and $\theta$ as separate nodes?

Similarly, i have the same confusion with equation 3. Expanding equation 2 i get $\int log(\frac{P(Y,X|\theta)}{q(X)q(\theta)}) q(X)q(\theta)d(\theta) + \int log(\frac{P(\theta)}{q(X)q(\theta))})q(X)q(\theta)d(\theta)$ . How is this equation the same as equation 3? The first term in equation 3 is missing a $q(\theta)$ in the denominator and the KL term should be with respect to $q(X,\theta)$ instead of $q(\theta)$ isnt it?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.