Shouldn't we sample from the output of variational auto-encoder?

Question

I know that the output of the VAE is the parameters of the data.
For example:
If the data follows normal distribution $X \sim \mathcal{N}(\mu,\sigma)$, the generative network should output $\mu$ and $\sigma$
If the data follows Bernoulli distribution $X \sim Bern(p)$, the network should output $p$ (the parameters of Bernoulli distribution)
And we should sample from that output to calculate the reconstruction loss.

All the example codes(that I saw) of VAE on the internet don't sample from the output of the decoder to calculate the reconstruction loss even if the output is normal distribution(they don't output $\sigma$).
They calculate the loss as if it's ordinary NN.
Is my understanding of VAE correct? or Is the practical implementation of VAE different from the theory?

Answer 1

There are two terms in the ELBO:

$$E_{z \sim q}[\log P(x|z)] - \text{KL}(q(z)||p(z))$$

We estimate the first term by sampling a single $z$ and computing $\log P(x|z)$.

Since the VAE models $x|z \sim \mathcal{N}(\mu, \sigma^2)$, where $\mu = f(z;\theta)$ for some decoder neural network $f$, and the log of the gaussian density is $-(\mu - x)^2$ (up to some constant factors and scaling), therefore this squared "reconstruction" loss is correct.

And we should sample from that output to calculate the reconstruction loss.

Not quite. You should sample from that output if you want to sample from the distribution modeled by the VAE, but we have shown here that the "reconstruction loss" between $\mu$ and $x$ is the correct loss to use.