# When generating samples using variational autoencoder, we decode samples from $N(0,1)$ instead of $\mu + \sigma N(0,1)$

Context: I'm trying to understand the use of variational autoencoders as generators. My understanding:

• During training, for an input point $$x_i$$ we want to learn latent $$\mu_i$$ and $$\sigma_i$$ and then sample $$z_i \sim N(\mu_i, \sigma_i)$$ and feed it to the decoder to get a reconstruction $$\hat{x}_i = \text{decode}(z_i)$$.
• But we can't do back propagation with sampling operator, so instead we reparametrize and use $$z_i = \mu_i + \sigma_i \epsilon$$ where $$\epsilon \sim N(0, 1)$$. Our reconstruction becomes $$\hat{x}_i = \text{decode}(\mu_i + \sigma_i \epsilon)$$.

However when we're done with training and ready to use it as generator, we sample $$z \sim N(0, 1)$$ and feed it to decoder: $$x_{sample} = \text{decode}(z)$$ .

The part that confuse me is that during training, the decode operation was done using $$\mu_i + \sigma_i \epsilon$$ which to my understanding this is using $$N(\mu_i, \sigma_i)$$ with different $$\mu_i$$ and $$\sigma_i$$ for each training example. However during the generation time, the decode operation is done (effectively) on $$\epsilon$$ alone from $$N(0, 1)$$. Why are we setting $$\mu = 0$$ and $$\sigma = 1$$ during generation (i.e. using $$z = 0 + 1 \cdot \epsilon$$)?

• – Edward B. Oct 1 '18 at 21:52

During training, we are drawing $$z \sim P(z|x)$$, and then decoding with $$\hat x = g(z)$$.
During generation, we are drawing $$z \sim P(z)$$, and then decoding $$x = g(z)$$.