# How should I intuitively understand the KL divergence loss in variational autoencoders? [duplicate]

This question already has an answer here:

I was studying VAEs and came across the loss function that consists of the KL divergence.

$$\sum_{i=1}^n \sigma^2_i + \mu_i^2 - \log(\sigma_i) - 1$$

I wanted to intuitively make sense of the KL divergence part of the loss function. It would be great if somebody can help me

## marked as duplicate by Michael Chernick, kjetil b halvorsen, Robert Long, Peter Flom - Reinstate Monica♦Oct 30 at 10:33

In VAE, let $$X$$ be the data we want to model, $$z$$ be latent variable, $$P(X)$$ be the probability distribution of data, $$P(z)$$ be the probability distribution of the latent variable and $$P(X|z)$$ be the distribution of generating data given latent variable
In the case of variational autoencoders, our objective is to infer $$P(z)$$ from $$P(z|X)$$. $$P(z|X)$$ is the probability distribution that projects our data into latent space. But since we do not have the distribution $$P(z|X)$$, we estimate it using its simpler estimation $$Q$$.
Now while training our VAE, the encoder should try to learn the simpler distribution $$Q(z|X)$$ such that it is as close as possible to the actual distribution $$P(z|X)$$. This is where we use KL divergence as a measure of a difference between two probability distributions. The VAE objective function thus includes this KL divergence term that needs to be minimized.
$$D_{KL}[Q(z|X)||P(z|X)] = E[\log {Q(z|X)} − \log {P(z|X)}]$$