In almost every code related to the VAE we generate x|z by simply getting it from the last layer of the decoder, e.g. using a sigmoid activation function. Why don't we sample from $\mathcal{N}(\mu, \Sigma)$? I see that $x|z \sim \mathcal(\mu, \Sigma)$, but when I actually try to sample from $\mathcal{N}(\mu, \Sigma)$ I get fuzzy results due to $\Sigma$ and it has to train a lot longer. Below two rows of stochastic MNIST samples can be seen, they move in time due to the recurrent nature, but the above samples are generated a normal distribution, so the loss is the negative log-likelihood of a Gaussian distribution, while the one below is found using a simple MSE loss and is the direct output of the decoder. Why is it that the samples from the Gaussian are a lot worse than the samples directly from the decoder output? There must be a reason every paper just uses the decoder output directly, but I don't see why. enter image description hereenter image description here



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.