For the vanilla autoencoder the structure is like this: 

[![enter image description here][1]][1]

It can be treated as a nonlinear extension of PCA, while for the variational autoencoder a mean and a standard deviation is added as a layer for each hidden variable in the middle layer: 

[![enter image description here][2]][2]

For the detailed difference please refer to [this answer](https://stats.stackexchange.com/a/421660/103153). 

> Should VAEs be even used for non-generative tasks? 

Yes, you can. The additional KL divergence(between variational distribution and the prior distribution/normal distribution) loss can be just seen a regularization and regularization can reduce the variance(at the risk of increasing bias). 

> If I were to use both models for embedding images, how would the embedding space differ on a structural level?

For the VAE the values of embedding would just be like samples from the normal distribution while that doesn't hold for the general autoencoder. 

Reference:     
[Intuitively Understanding Variational Autoencoders](https://towardsdatascience.com/intuitively-understanding-variational-autoencoders-1bfe67eb5daf)


  [1]: https://i.sstatic.net/O71SP.png
  [2]: https://i.sstatic.net/x7Jhu.png