# KL divergence of q(z|x) || p(z|x)

I don't completely understand KL divergence yet, and how the breakdowns are completely different for similar looking distributions.

In this explanation it says: $$KL(q_\lambda(z|x) \Vert p(z|x)) = E_q[\log \space q_\lambda(z|x)] - E_q[\log \space p(x,z)] + \log \space p(x)$$

The two distributions in the comparison look similar, both stating the probability distribution of $$z$$ given $$x$$, but in the first case it's the cross entropy ($$q$$ relative to $$q$$?), in the second case we're subtracting the cross entropy of $$p(x,z)$$ relative to $$q$$ plus $$\log \space p(x)$$.

Could someone explain why this problem should be solved like this?

• To what does the phrase "this problem" refer??
– whuber
Commented Dec 14, 2020 at 21:35
• There is no problem to solve. This identity follows from the decomposition of the conditional density $p(z|x)$ as$$p(z|x)=\dfrac{p(x,z)}{p(x)}$$ Commented Dec 15, 2020 at 5:47

By definition, we have that the $$\text{KL}$$ divergence of the probability distributions $$q_\lambda(z|x)$$ and $$p(z|x)$$ is given by: $$\text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) = \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z|x)) \right]$$ So it represents the expected value of the logarithmic difference between the probability distributions $$p(z|x)$$ and $$q_\lambda(z| x)$$ w.r.t. the probabilities given by $$q_\lambda(z| x)$$. This becomes useful to see how much one probability distribution differs from the other.
Given this, if we first apply Bayes' theorem to $$p(z|x)$$ (already noted in the comments), then use the quotient rule for logarithms and finally take advantage of the linearity of expectations, we get: \begin{align} \text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) &= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log\left(\frac{p(z,x)}{p(x)}\right) \right] \\ \\ &= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) + \log(p(x)) \right] \\ \\ &=\mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \mathbb{E}_q[\log(p(x))]\\ \\ \end{align}
Now, we have to realize that $$\log(p(x))$$ is a function of $$x$$, while $$q(z|x)$$ is a function of $$z$$ ($$x$$ is given/ constant). Knowing this we can calculate $$\mathbb{E}_q[\log(p(x))]$$ as follows (assuming continuous case):
$$\mathbb{E}_q[\log(p(x))] =\int q(z|x)\log(p(x)) dz =\log(p(x)) \int q(z|x) dz= \log(p(x))$$
This way, we have just arrived to the expression given by the authors: $$\text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right)= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \log(p(x))$$