# What is the factor equal to if the true and empirical distribution both are 0 for a configuration?

Suppose I want to calculate the relative entropy:

$$D(q||p)= \sum q(x)\log \frac{q(x)}{p(x)}$$

If, for some $x$, I have $q(x)=p(x)=0$, does the corresponding factor in the sum becomes $0$?

• meta.math.stackexchange.com/questions/5020/… Please use math typesetting.
– Sycorax
Aug 4, 2016 at 13:05
• If $q(x)=0$, why would you even include that term in the sum? By definition, it has no chance of occurring.
– whuber
Aug 4, 2016 at 13:15
• That's true, but I'm actually computing the KL distance of a Bayesian network, and I need to include such parameters in the code for the package to work. Aug 4, 2016 at 13:28
• Yes, but doesn't that consideration alone immediately tell you what the only possible correct answer can be?
– whuber
Aug 4, 2016 at 14:40

Wikipedia has the answer. Yes, the corresponding term becomes $0$.
The Kullback–Leibler divergence is defined only if $q(x)=0$ implies $p(x)=0$, for all $x$ (absolute continuity). Whenever $p(x)$ is zero the contribution of the x-th term is interpreted as zero because $\lim _{x\to 0}x\log(x)=0$
That is if both probabilities are $0$ you take the limit which, is $0$ for that term of the sum.