What is the encoding in variational autoencoders?

The question is actually less broad than it sounds. I generally do understand how variational autoencoders work.

From the encoding step we get four parts:

• mean $$\mu$$
• standard deviation $$\sigma$$
• random value from normal distribution $$\epsilon$$
• sample $$z$$

For the training process of the autoencoder, the decoder is fed with the sample $$z$$. If we wish to generate new samples, we manipulate the code that is fed to the decoder.

What output from the encoder to use for a pure encoding purpose (given a fully trained VAE)?

Please provide explanations and if possible a mathematical intuition / reasoning. If you have additional sources where this aspect is explained, please add links.

I'm deliberately and specifically using VAEs, that is not the question here.

• Isn't the point that the procedure implements variational Bayes, so you have a probability distribution over encodings from which you can sample freely? – Sycorax May 23 at 13:30
• @Sycorax That is why this questions comes up. Every material on VAEs focuses on the generative aspect but it is still an autoencoder and I would like to use it as such. – Spen May 23 at 13:34
• Doesn't that suggest to you that you're using the wrong tool for the job: you want an encoder, not a probability model? In any case, it seems like you could use a kludge like just fixing each element at its mode, which for a normal prior distribution is the mean vector $\mu$. Alternatively, you could marginalize over the distribution by taking many posterior samples for each input and pushing those through your downstream task and averaging/etc. – Sycorax May 23 at 13:55
• @Sycorax No, I'm specifically interested in properties that arise within the encoding (like disentanglement) of variational autoencoders. That was actually partially the question if using the mean vector $\mu$ would be a valid option. – Spen May 23 at 14:30