# When do these two definitions of KL-divergence match?

• Suppose $$P$$ and $$Q$$ are two distributions on a space $${\cal H}$$ (could be a subset of an infinite dimensional function space) with p.d.fs denoted by the same letter then one can define the $$KL$$ divergence between these distributions as, $$KL (Q || P) = \mathbb{E}_{h \sim Q} [ \log \frac{Q(h)}{P(h)} ] = \int_{{\cal H}} \log \frac{Q(h)}{P(h)} Q(h) d\mu(h)$$

• Suppose $$P$$ and $$Q$$ are two probability measures on $${\cal H}$$ s.t $$Q$$ is absolutely continuous w.r.t $$P$$ so that the Radon-Nikodym derivative exists. Suppose for any non-negative random variable with finite mean we define $$Ent[Z] = \mathbb{E} [ Z \log \frac{Z}{\mathbb{E}[Z]} ]$$. Then one can define the $$KL$$ divergence between these measures as, $$KL (Q || P) = Ent [ \frac{dQ}{dP} ]$$

My question is : When are these two notions the same?

In both cases we need $${\cal H}$$ to be a measure space and I am calling the measure defined on it as $$\mu$$. Kindly correct if I am writing anything wrong!

In the first equality I guess it is always implicit that one is having two continuous random variables valued in the measure space $${\cal H}$$ whose probability distributions are $$P$$ and $$Q$$.