In almost every code related to the VAE we generate x|z by simply getting it from the last layer of the decoder, e.g. using a sigmoid activation function. Why don't we sample from $\mathcal{N}(\mu, \Sigma)$? I see that $x|z \sim \mathcal(\mu, \Sigma)$, but when I actually try to sample from $\mathcal{N}(\mu, \Sigma)$ I get fuzzy results due to $\Sigma$ and it has to train a lot longer. Below two rows of stochastic MNIST samples can be seen, they move in time due to the recurrent nature, but the above samples are generated a normal distribution, so the loss is the negative log-likelihood of a Gaussian distribution, while the one below is found using a simple MSE loss and is the direct output of the decoder. Why is it that the samples from the Gaussian are a lot worse than the samples directly from the decoder output? There must be a reason every paper just uses the decoder output directly, but I don't see why. 
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Does this answer your question? Variational Autoencoder - How many Normal Distributions for Posterior $\endgroup$– shimaoNov 3, 2020 at 18:19
-
$\begingroup$ @shimaw assuming you have only the decoder part (essentialy a super resolution propblem) why can'y you sample directly from Gaussian pdf? $\endgroup$– partizanosJul 25, 2021 at 14:37
Add a comment
|