# Does variational auto-encoder output the variational distribution of the latent variable or the distribution of the input x?

In the simple case of mixture of gaussians(with known variance), we have 2 latent variables $$\mu$$ and $$z$$. In the vaiational auto-encoder, we assume that the model is infinite mixture of gaussians. If we assume that the variance is known then we have 2 latent variables $$\mu$$ and $$z$$. the encoder the responsible for calculating the variational distribution of $$z$$ (its mean and variance). and the decoder is responsible for calculating the variational distribution of the $$\mu$$ (it's mean and variance)
So the variance $$\sigma$$ that the generative network is outputting Is NOT the variance of $$x$$
My questions is
Is my viewpoint correct?
I read many articles on the internet that say $$P(x|z, w) = \mathcal{N}(\mu(z, w),\sigma(z, w))$$ where $$\mu(.)$$ and $$\sigma(.)$$ are the generative network with parameters $$w$$. Isn't that wrong?

PS: I know that training and results are the same in both cases

In a GMM, $$z$$ is a latent variable, but the $$\mu$$'s are a model parameter.
You seem to be trying to draw an analogy between GMMs and VAEs -- it's true you can think of a VAE as an infinite mixture model, where each component has $$\mu = f(z; \theta)$$ where $$f$$ is the decoder. Here $$\theta$$, not $$\mu$$, are the model parameters.
and the decoder is responsible for calculating the variational distribution of the $$\mu$$
No -- $$\mu$$ isn't a latent variable, so there's no need for any variational approximation for it.
Yes, $$x|z$$ has distribution $$\mathcal{N}(f(z;\theta))$$ -- this is simply how the VAE model is defined.
• I read before that we can treat GMM as a fully bayesian model and treat $\mu$ as a latent variable and calculate it using variational inference or we can just treat $\mu$ as a parameter and calculate it using EM. Maybe VAE is a variational EM not just variational inference(meaning that we don't treat all the unknown parameters $\mu$ and $z$ as latent variables). Am I right? Sep 12, 2019 at 7:19
• i'm not familiar with this "fully bayesian" form of GMM, so I can't comment on that. A VAE is not trained with EM. It is very much a variational inference method. $z$ is definitely a latent variable. Sep 12, 2019 at 8:28