# Kullback-Leibler divergence with sample data likelihood [duplicate]

I'm trying to get my head around the KL divergence in the context of the sample likelihood under two competing hypotheses, one optimal $H_0$ and one suboptimal $H_1$. Roughly speaking, I want to see the "difference in information" when $H_1$ is used instead of the better suited $H_0$ to describe the data. I write the data likelihood for each datum as $p(X_i \mid H_0)$ and $p(X_i \mid H_1)$. I could write the observed KL divergence as $$\sum_{i=1}^N p(X_i \mid H_0) \log \frac{p(X_i \mid H_0)}{p(X_i \mid H_1)}$$ However, both likelihoods over the sample data $X$ do not sum to one. Therefore, I'm not sure what the interpretation would be. I do consider the two situations (a) where I compare a good hypothesis $H_0$ to a number of suboptimal alternatives and (b) where I compare two independent pairs $(H_0, H_1)$ and $(H'_0, H'_1)$. All comparisons are done over the same sample data $X$.