New answers tagged


I believe the answer is simpler. In VAE, people usually use a multivariate normal distribution, which has covariance matrix $\Sigma$ instead of variance $\sigma^2$. That looks confusing in a piece of code but has the desired form. Here you can find the derivation of a KL divergence for multivariate normal distributions: Deriving the KL divergence loss for ...


Let $p(\theta \mid x)$ be the true posterior and $q_\phi(\theta)$ be the variational distribution (parameterized by $\phi$). The ELBO $\mathcal{L}(\phi)$ can be written as the difference between the log evidence and the KL divergence between the variational distribution and true posterior: $$\mathcal{L}(\phi) = \log p(x) - D_{KL} \Big( q_\phi(\theta) \...


Look at this tensorflow implementation. The author 1. build two dense nets to obtain $\vec \mu$ and $\vec \sigma$ at the last layer of the decoder (line 94); 2. use multivariate_normal.pdf function to compute reconstruction probability (line 161).

Top 50 recent answers are included