I'm watching Ian Goodfellow's introduction to generative models.

When he was introducing variational autoencoders at 22:29, he said:

Variational autoencoders are good at obtaining high likelihood, but they tend to produce lower quality samples, and in particular, the samples are relatively blurry.

My questions are:

  1. What does it mean by obtaining a high likelihood? Does it means that the density model that VAE learnt centres around those training samples? Is this the reason that causes VAE to produce low-quality samples?

  2. Why are the generated samples blurry? Is VAE averaging something when it is generating a new sample?

  3. When do we want high likelihood and when do we want high-quality samples?

  1. High likelihood meaning $E_{x \in X}[P(x;\theta)]$ is high, where $P(x;\theta)$ is the distribution over images that the VAE specifies. This corresponds to minimizing the KL-divergence between $P(x;\theta)$ and the ground-truth distribution, which is a good thing.

  2. The generated samples are blurry because VAE can only produce samples of the form $\mathcal{N}(f(z;\theta), \sigma^2 I)$. However, what people normally visualize as the output of a VAE is in fact just the decoder without the output noise: $f(z;\theta)$, which is the mean of that distribution. Obviously, viewing just the mean of a distribution will give you a blurred view of the actual output, and viewing the sample with added noise will provide something which looks excessively noisy. The reason for this is that adding independent gaussian noise is not a great model of how images are naturally generated, yet it is assumed implicitly by using MSE reconstruction loss.

  3. You pretty much always want both when training generative models.

| cite | improve this answer | |
  • $\begingroup$ How would you solve the blurriness? For example by changing the loss kind? $\endgroup$ – Nikaido May 19 at 15:07
  • 1
    $\begingroup$ @Nikaido search for "vqvae2" for the state of the art in VAE models. $\endgroup$ – shimao May 20 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.