Let's say we want to compare a vanilla Autoencoder to a Variational Autoencoder. The first one gives a deterministic output which basically represents the output with the highest likelihood.
When we compare the two approaches, is it legitimate to force the output Variational Autoencoder to the one with maximum likelihood, i.e. at $z=0$? Are there any references in literature that do this?
No if you stick to a Gaussian distribution. One alternative is to set the variance to be a very small number; thus, the sampling process is almost identical to picking the mean value.
If you don't stick to a Gaussian distribution, some variants of VAE have been proposed, such as VQ-VAE or adversarial encoders.
BTW, whether the Gaussian distribution in the VAE has variance or not, the model should be able to reach the highest likelihood as possible as it can, which is ensured by the objective function (maximizing the lower bound of data likelihood).
