I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) of test set examples (where x: observed data, z: latent) under the model, and higher log p(x) is better. However, how do you compute log p(x) on a test set data using a VAE? I thought that involves computing an intractable integral, but the paper includes many statistics of log p(x) under different VAE configurations without mentioning how they computed these values. Thanks in advance!
$\begingroup$
$\endgroup$
1
-
$\begingroup$ On Page 7, "All log-likelihood values were estimated as the mean of L_5000 on the test set." $\endgroup$– CP TaiJul 21, 2020 at 13:50
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The IWAE ELBO provides a tighter bound to the true log-likelihood $\log p(x)$. This bound gets tighter as the number of importance weighted samples $k$ increases.
Therefore, the authors chose a large enough $k$, in this paper $k$=5000, to approximate the true likelihood of the test data as $\widehat{\log p(x)}$. As such, one can assume that $\log p(x) \approx \widehat{\log p(x)} = \mathcal{L}_{k=5000}$.
As pointed out in the comment by @CP Tai, you can find more information about it in the paper from page 7 onwards.
