# Why don't we use the direct observation probability distribution in variational autoencoders?

In almost every code related to the VAE we generate x|z by simply getting it from the last layer of the decoder, e.g. using a sigmoid activation function. Why don't we sample from $$\mathcal{N}(\mu, \Sigma)$$? I see that $$x|z \sim \mathcal(\mu, \Sigma)$$, but when I actually try to sample from $$\mathcal{N}(\mu, \Sigma)$$ I get fuzzy results due to $$\Sigma$$ and it has to train a lot longer. Below two rows of stochastic MNIST samples can be seen, they move in time due to the recurrent nature, but the above samples are generated a normal distribution, so the loss is the negative log-likelihood of a Gaussian distribution, while the one below is found using a simple MSE loss and is the direct output of the decoder. Why is it that the samples from the Gaussian are a lot worse than the samples directly from the decoder output? There must be a reason every paper just uses the decoder output directly, but I don't see why.