2
$\begingroup$

The wasserstein_distance will be smaller the longer u_values and v_values are.

from scipy.stats import wasserstein_distance

def wassersteindist(n):    
    a = np.random.randn(n)
    b = np.random.randn(n)
    w = wasserstein_distance(a,b)
    return w
     
np.mean([wassersteindist(100) for r in range(1000)])
0.1786
np.mean([wassersteindist(1000) for r in range(1000)])
0.0579
np.mean([wassersteindist(10000) for r in range(1000)])
0.0180

Is there a way to calculate a normalized wasserstein distance with scipy?

EDIT: Let's say I 'm interested in comparing the distances from different individuals that happened to have a different amount of time points in their time series.

id1_a = np.random.randn(100)
id1_b = np.random.randn(100)

id2_a = np.random.randn(1000)
id2_b = np.random.randn(1000)

id1_dist = wasserstein_distance(id1_a, id1_b)
0.3539204677483332

id2_dist = wasserstein_distance(id2_a, id2_b)
0.0685546301855615

id1_dist is larger than id2_dist only because the vectors for id1 are shorter than for id2.

EDIT2:

With correlations I don't have the problem that they are consistently lower/higher for longer time series:

def corrr(n):    
    a = np.random.randn(n)
    b = np.random.randn(n)
    c = np.corrcoef(a,b)[0][1]
    return c

np.mean([corrr(100) for r in range(1000)])
0.0004

np.mean([corrr(1000) for r in range(1000)])
0.0012

np.mean([corrr(100) for r in range(1000)])
-0.0001

np.mean([corrr(1000) for r in range(1000)])
-0.0008
$\endgroup$
9
  • $\begingroup$ Why would you want to normalize it? And normalize it relative to what? $\endgroup$
    – jbowman
    Feb 17, 2023 at 21:28
  • $\begingroup$ @jbowman, I have edited my question with more information. Essentially, I would like to compare distances that were computed using a different amount of time points $\endgroup$
    – HappyPy
    Feb 17, 2023 at 21:34
  • $\begingroup$ The problem with this version of your previous question is that it lacks a statement of purpose and does not tell us much, if anything, about the probability model you are using to understand your data. That information will help us (or anyone else) answer both questions with suitable recommendations. $\endgroup$
    – whuber
    Feb 17, 2023 at 23:02
  • $\begingroup$ @whuber, I'm sorry for the lack of clarity with my questions. I have collected physiological data from several individuals (id) and grouped them according to a grouping factor (4 groups). Each id has two time series (x and y) that are the same length. However, one id could have 700 data points in x and y, whereas another could have 1400 datapoints. I could correlate the two signals for each id and compare the average correlation values among the groups, but I was simply looking for complementary measures of similarity. $\endgroup$
    – HappyPy
    Feb 18, 2023 at 0:13
  • $\begingroup$ @whuber, does my comment above help in any way? I'm not sure myself what the probability model could be. I was just looking for alternative measures of relationship between signals beyond correlations. $\endgroup$
    – HappyPy
    Feb 18, 2023 at 0:16

2 Answers 2

2
$\begingroup$

The fact that this distance is going down in your case is a good thing. In general, for two samples with empirical distributions $\mathbb P_n$ and $\mathbb Q_m$, we have have $$ W(\mathbb P_n, \mathbb Q_m) \to W(\mathbb P, \mathbb Q) $$ as $m, n \to \infty$. Here, $\mathbb P$ and $\mathbb Q$ are the population versions of the two distributions. In your simulations, the two samples are are coming from the same population $\mathbb P = \mathbb Q = N(0,1)$, hence $W(\mathbb P, \mathbb Q) = 0$. As $n,m \to \infty$, you should expect $W(\mathbb P_n, \mathbb Q_m)$ to approach zero.

For you time series, if they are coming from two different populations, $W(\mathbb P, \mathbb Q)$ will be nonzero, so you will converge to something nonzero for large samples.

In other words, you want $W(\mathbb P_n, \mathbb Q_m)$ to vary with $m$ and $n$ to get more accurate for larger samples. In general (in the 1-D case), $$\mathbb E[ W(\mathbb P_n, \mathbb Q_m) ] = W(\mathbb P,\mathbb Q) + O\Bigr(\frac1{\sqrt{n \wedge m}}\Bigl).$$ Normalizing $W(\mathbb P_n, \mathbb Q_m)$ to be invariant to the sample size does not make much sense, especially when $\mathbb P = \mathbb Q$.


In case you are curious where that rate comes from, first note that by triangle inequality $$W(\mathbb P_n, \mathbb Q_m) \le W(\mathbb P_n, \mathbb P) + W(\mathbb P, \mathbb Q) + W(\mathbb Q, \mathbb Q_m).$$ Then, use a result like that of Fournier and Guillin on the first and third terms.


EDIT1: The same problem is there with the absolute value of Pearson correlation coefficient:

import numpy as np

np.random.seed(1337)

def corrr(n):    
    a = np.random.randn(n)
    b = np.random.randn(n)
    c = np.corrcoef(a,b)[0][1]
    return c

for n in [10, 100, 1000, 10000]:
    print(np.mean([abs(corrr(n)) for r in range(1000)]))

which produces the following output:

0.2709428042954994
0.07715058968038786
0.023744887030041146
0.008094465791386103
$\endgroup$
6
  • $\begingroup$ thanks for your answer. So if I understood your answer, comparing wasserstein distances that were computed with vectors of different lengths do not make sense? $\endgroup$
    – HappyPy
    Feb 19, 2023 at 23:09
  • $\begingroup$ That is not quite what I said. Looking at your comments on the original post, you seem to be OK with computing correlations when the two times series have different sizes (?) This is very much similar. If you are OK with that, you can be OK with this too. (Sample correlation is an estimate of the population correlation. Sample Wasserstein is an estimate of the population Wasserstein.) $\endgroup$
    – passerby51
    Feb 20, 2023 at 5:57
  • $\begingroup$ Thanks @passerby51. But with correlations I'm not having the problem that individuals with longer time series have consistently lower/higher correlation values than those with shorter time series (see Edit2). But with wasserstein distance there will be a bias. In the extreme case that all subjects in group B have longer time series than subjects in group A, I will for sure find a difference between the two groups just due to the fact that the time series for subjects in group B is longer... This won't happen with correlations. $\endgroup$
    – HappyPy
    Feb 20, 2023 at 9:03
  • $\begingroup$ @HappyPy, You have the same problem with correlation, but since correlation is signed you are not seeing it (Wasserstein is always nonnegative). Try calculating the absolute value of the correlation. See the updated code in my edited response. On a side note, there is always a way to normalize a statistic in statistics, via figuring out its null distribution. As long as you have an idea what your null is, there is a way. $\endgroup$
    – passerby51
    Feb 22, 2023 at 19:09
  • $\begingroup$ thanks for the edit, I had never thought of correlations in that way. Then is it possible to construct the null distribution based on my actual data? Should I post another question about this? $\endgroup$
    – HappyPy
    Feb 22, 2023 at 21:46
0
$\begingroup$

If you're looking for a metric that can be used to estimate the distance between two sets of data that is not sensitive to the size of each sample or to the scale of the data, you might want to consider the Kolmogorov-Smirnov statistic, which is implemented in scipy here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.