What's the difference between random and deterministic encoder in autoencoders? I read this paper "Wasserstein Auto-Encoders", and they mention deterministic encoder and random encoder but without stating the difference between them. How can we tell the difference?
 A: So I read the same paper, and had the same exact question. I actually came across this post when trying to figure it out. It's not an easy question to answer because the original paper, and other papers I've read referring to deterministic encoders, never explain what they are. But I think I can answer this question now.
A deterministic encoder is just a mapping F:X -> Z. Basically, your encoder behaves like the encoder in a plain old AE.
The answer makes more sense when you think about what VAE does. In VAE, your encoder learns the parameters of a distribution for the latent space Z, and is therefore nondeterministic. We can then feed samples from this distribution in Z to our decoder P(X|Z), thus we have a generative model. Our decoder is a generator. We can generate samples we get from Z that weren't encoded from our original dataset X.
With Wasserstein Autoencoders (WAE), we have to option to use a deterministic encoder. So I believe our model is no longer generative if we do this, it's just a mapping F:X -> Z. But it's computationally simpler, thus faster. But we don't have an AE exactly, we still have the following benefits of WAE:


*

*The option to enforce a non-Gaussian prior on the latent space. This is crucial if your latent space doesn't follow a Gaussian. We can do this by replacing the regularization term in the WAE objective function with Maximum Mean Discrepancy (MMD) or JS Divergence. The latter option makes our model adversarial (we have to use a discriminator to enforce the prior).

*The expectation in the regularization term is over the aggregated posterior as opposed to the posterior as in VAE. This helps prevent the posterior collapse problem experienced in VAE, which is when the posterior becomes independent of the input. Basically, the decoder learns to ignore the input z, and learns a locally optimal solution, which is simply the prior distribution. So simply put, it learns to mimic the prior as a solution.
This was my takeaway after doing some research and experimenting with WAE on a dataset. I'd love to hear someone else with experience chime in.
