According to VAE paper : https://arxiv.org/pdf/1906.02691 (eq 2.10, page 21) VAE loss contain the expectation term. If I understand correctly, we need sample more than 1 result from an input x and average it's loss to satisfy the expectation term. But I find many implementation that just only sample one result, but why?
2
-
2$\begingroup$ Though only using one sample will have a high variance, its expectation is still correct; this is a broader issue beyond VAEs and you may want to use search terms like "Monte Carlo Approximation", "Stochastic Gradient Descent" or "Finite Sum Problem". $\endgroup$– John MaddenCommented Sep 22 at 18:55
-
$\begingroup$ @JohnMadden it's very helpful, thanks a lot. $\endgroup$– ManhCommented Sep 22 at 18:57
Add a comment
|