variational inference with KL i am self-studying variational inference - and in Murphy's book "A probabilistic perspective on machine learning" it is discussed that minimizing the forward KL divergence (which is stated to be zero-avoiding for q) $$KL (p||q) = \sum\limits_{x} p(x)log \frac {p(x)}{q(x)}$$ gives different results than minimizing the reverse KL divergence $$KL (q||p) = \sum\limits_{x} q(x)log \frac {q(x)}{p(x)}$$ (which is stated to be zero-forcing for q). It is subsequently linked to the diagrams per below. I was hoping whehter someone can share the intuition on how the formulas for reverse and forward KL lead to the different approximations per below. 

 A: The KL divergences can be seen as a product of a weighting function $w(x)$ and a penalty function $g(x)$, i.e. $KL(q||p) = \sum_x w(x)g(x)$ with $w(x) = q(x)$ and $g(x) = \log\frac{q(x)}{p(x)}$ in the case of the reverse KL divergence. Whenever the weighting function is close to zero, i.e. $w(x) \approx 0$, the product of $w(x)g(x)$ is also close to zero and the value of the penalty function $g(x)$ does not contribute to the KL divergence no matter how large it is. 
Consider first the case of reverse KL divergence. When $q(x)$ is close to zero, the penalty term $g(x) = \log\frac{q(x)}{p(x)}$ is ignored and therefore the reverse KL divergence ''ignores'' the portion of $p(x)$, which is not covered by $q(x)$. On the other hand, when $q(x)$ has significant mass and $p(x)$ is close to zero the penalty term $g(x)$ will be large. Therefore the reverse KL divergence discourages situations where $q(x)$ is high and $p(x)$ is small leading to the ''zero-forcing''-effect. 
We can now make a similar analysis of the ''forward'' KL divergence. Now the weighting function corresponds to the target distribution $p$, i.e. $w(x) = p(x)$. Thus, when $p(x) \approx 0$ the value of the penalty $g(x) = \log\frac{p(x)}{q(x)}$ is largely ignored. Thus, there is almost no cost of having large $q(x)$ when $p(x)$ is small. On the other hand we see that when $p(x)$ has significant mass and $q(x)$ is small the contribution to the KL divergence is large. Combining these two properties leads to the ''zero-avoiding'' property.
