Should VAEs be even used for non-generative tasks? If I were to use both models for embedding images, how would the embedding space differ on a structural level?
For the vanilla autoencoder the structure is like this:
It can be treated as a nonlinear extension of PCA, while for the variational autoencoder a mean and a standard deviation is added as a layer for each hidden variable in the middle layer:
For the detailed difference please refer to this answer.