Decoder of variational autoecnoder From various explanation (e.g. https://www.youtube.com/watch?v=uaaqyVS9-rM), the decoder part of variational autoencoder corresponds to p(x|z) (or p(x_i|z_i)). However, if we fix z (sample a particular z from latent space), then the output is fixed instead of a distribution. So why does it corresponds to p(x|z)? Thanks in advance.
 A: In VAEs, the decoder is implemented using a neural net. The network takes a realization $z$ of the latent variable as input, and outputs parameters of the conditional distribution $p(x \mid z)$ over the observed variable $x$.
For example, if $x = [x_1, \dots, x_d]^T$ is a binary vector, it's common to define $p(x \mid z)$ as a factorized multivariate Bernoulli distribution, where each element $x_i$ is conditionally independent of the others. In this case, the decoder network typically has a sigmoidal output layer $o$, where each unit $o_i$ specifies the probability that $x_i=1$:
$$p(x \mid z) = \prod_i \operatorname{Bernoulli}(x_i \mid o_i)$$
If $x \in \mathbb{R}^d$ is continous, it's common to define $p(x \mid z)$ as an isotropic Gaussian distribution. In this case, the decoder network typically has a linear output layer $o$ with $d+1$ units, where the first $d$ units specify the mean, and the last unit specifies the log variance:
$$p(x \mid z) = \mathcal{N}(x \mid \mu, \sigma^2 I)$$
$$\mu = [o_1, \dots, o_d]^T \quad
\sigma^2 = \exp(o_{d+1})$$
Using the log variance lets us avoid constraints on the optimization problem, since exponentiating it always produces a valid, positive value for the variance.
