2
$\begingroup$

I've been reading this paper "Understanding disentangling in beta-VAE" and there was one part I was confused about. https://arxiv.org/abs/1804.03599 (paper link)

On Figure 4, it shows that if the KL-divergence loss between the latent posterior distribution and the latent prior distribution is close to, or equal to 0, varying the values in that latent dimension doesn't produce any changes in the generated images. Why is that?

My understanding is, if the KL-divergence loss is 0, we can assume that that latent dimension's distribution is an isotropic unit gaussian distribution, so it has 0 mean and 1 variance. Why does the fact that it's isotropic unit gaussian mean the generated images don't change?

$\endgroup$
1
$\begingroup$

If the KL-divergence between the posterior $p(z|x)$ and the prior $p(z)$ is close to 0, then $z$ is almost independent of the data point $x$. Since $x$ depends minimally on $z$, changes in $z$ lead to small changes in $x$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.