# Calculate probability vector from sample data

I'd like to compare two distributions using Jensen-Shannon Divergence metric. To do this, I need two probability vectors to plug into distance.jensenshannon(p, q). From the scipy.spatial documentation.

scipy.spatial.distance.jensenshannon(p, q, base=None)[source]

Parameters:

p(N,) array_like left probability vector

q(N,) array_like right probability vector

Question

How can I calculate probability vectors from sample data?

Example:

from scipy.spatial import distance
from scipy import stats
import numpy as np

x1 = np.random.normal(size=100)
x2 = np.random.normal(size=100)

p =

q =

jsd_metric = distance.jensenshannon(p, q)


Can I accomplish this using scipy.stats.norm.pdf()?

p = scipy.stats.norm.pdf(x1)
q = scipy.stats.norm.pdf(x2)

• There are at least four distributions in evidence: the Normal distributions used to generate data and the empirical distributions of the data. Exactly which distributions do you wish to compare? – whuber Jun 28 '19 at 21:38
• I want to compare x1 and x2. So I want to compare the empirical distribution of x1 and x2. – Amstell Jun 28 '19 at 21:41

I don't really know this test but if I base my code on that formula:

where M = (P+Q)/2 and D(Q|M) the KLD between Q and M (same for D(P|M) so on python I do this:

import numpy as np
import scipy.stats

x1 = np.random.normal(size=100)
x2 = np.random.normal(size=100)

p = scipy.stats.norm.cdf(x1)
q = scipy.stats.norm.cdf(x2)

m = (p + q) / 2

divergence = (scipy.stats.entropy(p,m) + scipy.stats.entropy(q,m)) / 2

distance = np.sqrt(divergence)

• Thanks! Why scipy.stats.norm.cdf()? Why not .pdf() – Amstell Jun 29 '19 at 3:14
• I don't think it impacts the results that much, I read "probability distribution" so I used CDF instinctively but a PDF is also a description of a probability distribution. I'm not familiar with this test but since you only need "probability vector", I think either PDF or CDF are good enough. – josef_joestarr Jun 29 '19 at 16:05