I'd like to compare two distributions using Jensen-Shannon Divergence metric. To do this, I need two probability vectors to plug into distance.jensenshannon(p, q). From the scipy.spatial documentation.

scipy.spatial.distance.jensenshannon(p, q, base=None)[source]


p(N,) array_like left probability vector

q(N,) array_like right probability vector


How can I calculate probability vectors from sample data?


from scipy.spatial import distance
from scipy import stats
import numpy as np

x1 = np.random.normal(size=100)
x2 = np.random.normal(size=100)

p = 

q = 

jsd_metric = distance.jensenshannon(p, q)

Can I accomplish this using scipy.stats.norm.pdf()?

p = scipy.stats.norm.pdf(x1)
q = scipy.stats.norm.pdf(x2)
  • $\begingroup$ There are at least four distributions in evidence: the Normal distributions used to generate data and the empirical distributions of the data. Exactly which distributions do you wish to compare? $\endgroup$ – whuber Jun 28 '19 at 21:38
  • $\begingroup$ I want to compare x1 and x2. So I want to compare the empirical distribution of x1 and x2. $\endgroup$ – Amstell Jun 28 '19 at 21:41

I don't really know this test but if I base my code on that formula:

enter image description here

where M = (P+Q)/2 and D(Q|M) the KLD between Q and M (same for D(P|M) so on python I do this:

import numpy as np
import scipy.stats

x1 = np.random.normal(size=100)
x2 = np.random.normal(size=100)

p = scipy.stats.norm.cdf(x1)
q = scipy.stats.norm.cdf(x2)

m = (p + q) / 2

divergence = (scipy.stats.entropy(p,m) + scipy.stats.entropy(q,m)) / 2

distance = np.sqrt(divergence)
  • $\begingroup$ Thanks! Why scipy.stats.norm.cdf()? Why not .pdf() $\endgroup$ – Amstell Jun 29 '19 at 3:14
  • $\begingroup$ I don't think it impacts the results that much, I read "probability distribution" so I used CDF instinctively but a PDF is also a description of a probability distribution. I'm not familiar with this test but since you only need "probability vector", I think either PDF or CDF are good enough. $\endgroup$ – josef_joestarr Jun 29 '19 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.