2
$\begingroup$

I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me:

GANs and VAEs have demonstrated impressive performance results on challenging tasks such as learning distributions of natural images. However, several issues limit their application in practice. Neither allows for exact evaluation of the probability density of new points.

Maybe I misinterpret it somehow... But can someone explain to me why VAE (or GAN to this matter) cannot evaluate probability density of new points? Isn't it the whole idea of estimating PDF's parameters, so we can evaluate it at any point we want, i.e. new points? Or do you think the author was making emphasis on "exact evaluation" part?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

They're some sort of generative models, learning the PDF so that they can sample from it. This is achieved by having random latent representations and mapping them to the relevant domain. So, you can easily generate new samples, but, this is not the same thing as calculating the probability of that sample. The learnt parameters belong to the neural network, not the PDF itself directly. There is no explicit calculation of the probability. To do that, you may need to resort to Monte Carlo methods.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Oh, in other words latent space distribution has nothing to do with p(x) because in order to generate new sample x_hat we sample from latent distribution Z and map it to domain X using decoder of VAE, i.e. we use p(x|z). And you're saying that in order to obtain p(x) we can use Monte Carlo methods. Right? $\endgroup$
    – Pavel Podlipensky
    Aug 28, 2021 at 23:42
  • 1
    $\begingroup$ In principle, yes, because we have a way to sample the distribution. But it may be quite expensive. $\endgroup$
    – gunes
    Aug 28, 2021 at 23:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.