When doing variational inference, due to intractability we typically maximize the evidence lower bound (ELBO) instead of minimizing Kullback-Leibler divergence (KLD) between our approximate and exact posterior. Assuming that we can compute gradients of KLD, my question is the following: are gradients of ELBO and KLD evaluated at the same values of our variational parameters pointing to the same direction (modulo sign) or can they be different.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Let $p(\theta \mid x)$ be the true posterior and $q_\phi(\theta)$ be the variational distribution (parameterized by $\phi$). The ELBO $\mathcal{L}(\phi)$ can be written as the difference between the log evidence and the KL divergence between the variational distribution and true posterior:
$$\mathcal{L}(\phi) = \log p(x) - D_{KL} \Big( q_\phi(\theta) \parallel p(\theta \mid x) \Big)$$
Take the gradient of both sides w.r.t. the variational parameters. The log evidence is constant, so $\nabla_\phi \log p(x) = 0$ and:
$$\nabla_\phi \mathcal{L}(\phi) = -\nabla_\phi D_{KL} \Big( q_\phi(\theta) \parallel p(\theta \mid x) \Big)$$
So, the gradients of the ELBO and KL divergence are opposites.
answered Oct 24, 2019 at 21:45
-
$\begingroup$ Does your notation $q_\phi(\theta)$ imply that $q$ is independent of the data $x$? (I'm also learning about this, hope you don't mind the question.) I thought that $q$ was updated based on $x$ $\endgroup$– 900edgesMar 15, 2021 at 15:39
-
1$\begingroup$ @900edges The way I've written things here, $x$ is the data, $\theta$ are model parameters, and the approximate posterior $q_\phi(\theta)$ is completely determined by the variational parameters $\phi$. The variational parameters are learned from the data. In particular, we choose them to maximize the ELBO $\mathcal{L}(\phi)$, which is a lower bound on the log marginal likelihood $\log p(x) = \int p(x \mid \theta) p(\theta) d\theta$. So, $q_\phi(\theta)$ is indeed updated based on $x$. $\endgroup$ Mar 15, 2021 at 17:19