KL divergence of q(z|x) || p(z|x)

Question

I don't completely understand KL divergence yet, and how the breakdowns are completely different for similar looking distributions.

In this explanation it says: $$KL(q_\lambda(z|x) \Vert p(z|x)) = E_q[\log \space q_\lambda(z|x)] - E_q[\log \space p(x,z)] + \log \space p(x)$$

The two distributions in the comparison look similar, both stating the probability distribution of $z$ given $x$, but in the first case it's the cross entropy ($q$ relative to $q$?), in the second case we're subtracting the cross entropy of $p(x,z)$ relative to $q$ plus $\log \space p(x)$.

Could someone explain why this problem should be solved like this?

Answer 1

By definition, we have that the $\text{KL}$ divergence of the probability distributions $q_\lambda(z|x)$ and $p(z|x)$ is given by: $$\text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) = \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z|x)) \right]$$ So it represents the expected value of the logarithmic difference between the probability distributions $p(z|x)$ and $q_\lambda(z| x)$ w.r.t. the probabilities given by $q_\lambda(z| x)$. This becomes useful to see how much one probability distribution differs from the other.

Given this, if we first apply Bayes' theorem to $p(z|x)$ (already noted in the comments), then use the quotient rule for logarithms and finally take advantage of the linearity of expectations, we get: $$\begin{align} \text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) &= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log\left(\frac{p(z,x)}{p(x)}\right) \right] \\ \\ &= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) + \log(p(x)) \right] \\ \\ &=\mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \mathbb{E}_q[\log(p(x))]\\ \\ \end{align}$$

Now, we have to realize that $\log(p(x))$ is a function of $x$, while $q(z|x)$ is a function of $z$ ($x$ is given/ constant). Knowing this we can calculate $\mathbb{E}_q[\log(p(x))]$ as follows (assuming continuous case):

$$\mathbb{E}_q[\log(p(x))] =\int q(z|x)\log(p(x)) dz =\log(p(x)) \int q(z|x) dz= \log(p(x))$$

This way, we have just arrived to the expression given by the authors: $$ \text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right)= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \log(p(x))$$