6
$\begingroup$

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) of test set examples (where x: observed data, z: latent) under the model, and higher log p(x) is better. However, how do you compute log p(x) on a test set data using a VAE? I thought that involves computing an intractable integral, but the paper includes many statistics of log p(x) under different VAE configurations without mentioning how they computed these values. Thanks in advance!

$\endgroup$
1
  • $\begingroup$ On Page 7, "All log-likelihood values were estimated as the mean of L_5000 on the test set." $\endgroup$
    – CP Tai
    Jul 21, 2020 at 13:50

1 Answer 1

2
$\begingroup$

The IWAE ELBO provides a tighter bound to the true log-likelihood $\log p(x)$. This bound gets tighter as the number of importance weighted samples $k$ increases.

Therefore, the authors chose a large enough $k$, in this paper $k$=5000, to approximate the true likelihood of the test data as $\widehat{\log p(x)}$. As such, one can assume that $\log p(x) \approx \widehat{\log p(x)} = \mathcal{L}_{k=5000}$.

As pointed out in the comment by @CP Tai, you can find more information about it in the paper from page 7 onwards.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.