KL divergence of q(z|x) || p(z|x) I don't completely understand KL divergence yet, and how the breakdowns are completely different for similar looking distributions.
In this explanation it says:
$$KL(q_\lambda(z|x) \Vert p(z|x)) = E_q[\log \space q_\lambda(z|x)] - E_q[\log \space p(x,z)] + \log \space p(x)$$
The two distributions in the comparison look similar, both stating the probability distribution of $z$ given $x$, but in the first case it's the cross entropy ($q$ relative to $q$?), in the second case we're subtracting the cross entropy of $p(x,z)$ relative to $q$ plus $\log \space p(x)$.
Could someone explain why this problem should be solved like this?
 A: By definition, we have that the $\text{KL}$ divergence of the probability distributions $q_\lambda(z|x)$ and $p(z|x)$ is given by:
$$\text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) = \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z|x)) \right]$$
So it represents the expected value of the logarithmic difference between the probability distributions $p(z|x)$ and $q_\lambda(z| x)$ w.r.t. the probabilities given by $q_\lambda(z| x)$. This becomes useful to see how much one probability distribution differs from the other.
Given this, if we first apply Bayes' theorem to $p(z|x)$ (already noted in the comments), then use the quotient rule for logarithms  and finally take advantage of the linearity of expectations, we get:
$$\begin{align}
 \text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right) &= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log\left(\frac{p(z,x)}{p(x)}\right) \right]  \\
\\
&= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) + \log(p(x)) \right] \\
\\
&=\mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \mathbb{E}_q[\log(p(x))]\\
\\
\end{align}$$
Now, we have to realize that $\log(p(x))$ is a function of $x$, while $q(z|x)$ is a function of $z$ ($x$ is given/ constant). Knowing this we can calculate $\mathbb{E}_q[\log(p(x))]$ as follows (assuming continuous case):
$$\mathbb{E}_q[\log(p(x))] =\int q(z|x)\log(p(x)) dz =\log(p(x)) \int q(z|x) dz= \log(p(x))$$
This way, we have just arrived to the expression given by the authors:
$$ \text{KL}\left(q_\lambda(z| x)\,\Vert\, p(z|x)\right)= \mathbb{E}_q\left[\log(q_\lambda(z|x)) - \log(p(z,x)) \right] + \log(p(x))$$
