# What is the Kullback–Leibler divergence result for this example

Suppose we have a set of tags t1 to t6 as below, also Ua and Ub are user A and user B and they have used tags as below:

Reference set = {t1, t2, t3, t4, t5, t6}
Ua = {t1, t2, t4, t1}
Ub = {t1, t5, t6}
P(A|B) = P(A intersect B) / P(B) = 1/3


I have computed the "conditional probability" between 2 users and it is 1/3. But when I compute the Kullback–Leibler divergence for the above example, I get a negative answer(-0.135)! while according to Wikipedia : The Kullback–Leibler divergence is always non-negative

I computed this value according to 2 below equations in an article:

In a KL-divergence-based approach, using the user tags, trust information from user ua to user ub, , can be defined as equation (1)

where T(ua) is a set of tags annotated by user ua, and ft(ua, s) and ft(ub, s) are the probability mass functions of users ua and ub over the tags, respectively, and can be defined as equation (2)

Here, nt(ua, s) and nt(ua, r) are the distribution of tags s and r of user ua, respectively. The system evaluates the trust relationship between two users using KL divergence.

This is 2 equations and my work to compute the result:

• How do you compute the Kullback-Leibler divergence? This could be the explanation! – Xi'an Sep 19 '15 at 14:45
• KL is non-negative for probability measures. Did you properly normalize your distributions to have total mass of 1? – Memming Sep 19 '15 at 15:08
• How should i normalize the above example? can you tell me what is the KL-divergence result for this example? – mdf Sep 19 '15 at 15:33
• Why can't you tell us the way you computed it? For instance, include the R code you used. – Xi'an Sep 19 '15 at 16:06
• I edited the post and added the way I have computed the result. – mdf Sep 19 '15 at 16:59