1
$\begingroup$

I have been reading up a bit on generative models particularly trying to understand the math behind VAE. While looking at a talk online, the speaker mentions the following definition of marginal likelihood, where we integrate out the latent variables:

$$ p(x) = \int p(x|z)p(z)dz $$

Here we are marginalizing out the latent variable denoted by z.

Now, imagine xare sampled from a very high dimensional space like space of all possible images of a given size but the prior p(z)is a unit Gaussian distribution. I am trying to understand why this would be difficult to evaluate considering p(z)is one dimensional.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

$z$ is still fairly high dimensional, typical values might range between 16 and 1000. This is still much lower than the dimension of $x$, which might be on the order of 10000 to 1 million. $p(z)$ is a standard multivariate gaussian, with identity covariance.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for replying. But in this talk there is only a single latent variable where the prior is a unit Gaussian. I am wondering why the integral is then still difficult to compute. This also comes up in this Stanford talk (youtube.com/watch?v=5WoItGTWV54&t=2525s) at 31.12. However, the speaker does not really specify what the problem is with this integral. $\endgroup$
    – Luca
    Nov 21, 2021 at 21:24
  • $\begingroup$ @Luca I don't see anywhere where the lecturer claims $z$ is only one dimensional. I agree in that case, the integral can be tractably evaluated. $\endgroup$
    – shimao
    Nov 21, 2021 at 21:29
  • $\begingroup$ Thanks for clarifying. I must have got myself confused. $\endgroup$
    – Luca
    Nov 21, 2021 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.