# VAE with mixture of gaussian prior

I try to understand this paper where they try to use a mixture of Gaussian as a prior, instead of the standard gaussian. There are several things unclear to me though:

1. They say that they set $$\pi_k = \frac{1}{K}$$ and draw $$z$$ from Cat($$\pi$$). But later in equation 5 they parameterize $$p_\beta(z_k = 1|x, w)$$ by a neural network, and also condition it on the inputs. How does that fit together?
2. Also in equation 5 they calculate a KL divergence. What is this KL divergence explicitly though? How to optimize it?
3. They don't write how to optimize the $$z$$-prior term in equation 4. Is there a closed form solution similar to the $$w$$-prior term? Or how would one do that?

## 1 Answer

1. Cat($$\pi$$) is the prior distribution on $$z$$. $$p_\beta(z|x,w)$$ is the variational approximation of the posterior of $$z$$.

2. It's the KL between the variational approximation of the posterior of $$x$$ (technically it's just one factor of the mean field variational approximation) and the conditional prior of $$x$$. These are both normal distributions, so computing the KL is easy and differentiable.

3. I believe $$p(z)$$ is just Cat($$\pi$$), so the KL can be written in closed form.

• So 2. simply with the reparametrization trick? And for 3. what is the closed form of the KL? – Luca Thiede Nov 30 '18 at 19:31