Questions tagged [wasserstein]
The Wasserstein metric or Earth Movers Distance is a distance function between probability distributions.
47
questions
0
votes
0
answers
20
views
Can we train normalizing flows with Wasserstein distance?
To train flow based models, you usually either use forward or reverse kl as your loss function. My question is, can you use wasserstein distance directly as your loss function to replace kl? I have ...
- 101
0
votes
0
answers
7
views
Problem for training Wasserstein GAN
I'm trying to train a Wasserstein GAN to guess sparse one-hot encoded matrices (0/1), in particular I've reimplemented the same architecture proposed in this paper.
The problem, as you can see, is ...
- 427
0
votes
0
answers
11
views
Wasserstein Adversarial Imitation Learning
in the paper, its mentioned that the authors used L2 regularization during training as shown below.
−1/4ε(r(y) − r(x) − d(x, y))2
but it is not clear to me how to implement it, any hints?.
- 1
0
votes
0
answers
10
views
Getting rid of the additive degree of freedom for discriminators of WGAN-GP's
Setting: Discriminators in WGAN-GP's are trained to minimise the following loss functional over functions D:
Here
I have been playing around with training a critic (simple convolutional network ...
- 101
0
votes
0
answers
21
views
Estimating empirically the Wasserstein distance
I have a dataset of the form $\{x_i,y_i,y_i'\}$, where $y_i\sim p(\cdot|x_i)$ and $y_i'\sim q(\cdot|x_i)$, while $x_i$ itself has a distribution $d(\cdot)$
Is there I way to estimate
$$\mathbb E_{x\...
- 101
0
votes
0
answers
48
views
I need help understanding the meaning of the loss values of a WGAN with Gradient Penalty
I am currently working on training a Auxiliary Classifier Wasserstein GAN with Gradient Penalty. I based my implementation off of https://keras.io/examples/generative/wgan_gp/ (to which I added the ...
1
vote
1
answer
291
views
Wasserstein distance and Kolmogorov-Smirnov statistic as measures of effect size
So I've been dealing with 2 sample hypothesis testing with very large samples (around 20,000s each). Whenever I test for the equality of distribution I always reject the null hypothesis, even though ...
- 23
0
votes
0
answers
25
views
Measure for evaluating a density estimation procedure
Given an implementation of a multivariate density estimation scheme, what would be a suitable measure to evaluate the accuracy of the procedure?
I am currently evaluating the procedure using three ...
- 964
0
votes
0
answers
141
views
Interpretation of upper bound on the Wasserstein Distance
I am trying to interpret the 2-Wasserstein distance and the upper bound on it. Let's say I have 2-Wasserstein distance between two distributions to be $x$, and I have an upper bound on it which gives ...
- 185
1
vote
1
answer
335
views
Intuition on Wasserstein Distance
I've been trying to familiarize myself with the Wasserstein distance and saw this answer on StackExchange by @antike that at first made a lot of sense, but then it didn't (to me, of course).
In the ...
- 163
0
votes
1
answer
77
views
How can one compute the "average" of a dataset of histograms that minimizes the mean Earth Mover's Distance between all data points and average?
It is my understanding that when the distance metric is euclidean distance, the mean of a dataset minimizes the average distance between all data points and the computed "mean".
In the case ...
1
vote
0
answers
35
views
What is this probability distribution?
Thank you in advance for any suggestions or feedback.
I have a discrete 1D probability distribution represented as a vector $\textbf{p}$, $p_i = p(x_i)$.
I am interested in finding the Wasserstein (...
1
vote
1
answer
695
views
When is my Wasserstein GAN-GP overfitting?
I have a hard time interpreting the WGAN-GP losses attached. At which epoch is D and/or G overfitting? The quality improves a lot overtime, yet the generator loss at later epochs does not appear on ...
0
votes
0
answers
286
views
Wasserstein distance for categorical data? Relationship to TVD?
Is the Wasserstein distance applicable for categorical data? e.g. if we have the distribution of different coloured balls in two bags,
...
1
vote
0
answers
71
views
Reference for Wasserstein distance for generalized exponential family and generalized extreme value probability distributions
The Wikipedia page gives the analytical results for computing the Wasserstein metric between two multivariate normal distributions, and I thought it used to give results for several other commonly ...
- 297
-1
votes
1
answer
47
views
K-Lipschitz for the Wasserstein GANs
I am trying to follow this blog for Wasserstein loss for Generative Adversarial Networks:
From GAN to WGAN.
Actually, I am trying to follow the logic behind the K-Lipschitz continuity. This post in ...
- 75
1
vote
1
answer
17
views
How to check whether a partitioning is more balanced?
For example consider the following two partitions that add approximately to 100 percent.
Partition 1 : 41% 36% 14% 5% 2% 1% <1% <1%
Partition 2 : 82% 10% 4% 3% <1% <1% <1% <1%
...
- 180
0
votes
0
answers
43
views
Horizontal vs Vertical optimal transport of probability distributions
Optimal transport has a primal and dual form. My question is: Is the primal formulation of OT intended for horizontal movement of probability mass, whereas the dual formulation is more geared towards ...
- 3,067
0
votes
0
answers
111
views
Does Wasserstein distance require the source and target distributions to have the same mass?
If we minimize the Wasserstein loss,
$$W_1 (P_S, P_T) = \underset{\gamma \in \Pi}{\text{min }} \sum_{x,y} |x-y|\gamma(x,y)$$
which means we are looking for the coupling $\gamma(x,y)$ that minimizes ...
- 3,067
0
votes
0
answers
38
views
If entropy is the underlying measure for KL-divergence, what is the underlying measure for the Wasserstein distance?
If entropy is the basis measure underlying KL-divergence (aka relative entropy), what is the basis measure underlying the Wasserstein distance?
- 3,067
1
vote
0
answers
77
views
Properties of Sinkhorn distance
I am reading the paper by Cuturi http://www.marcocuturi.net/Papers/cuturi13sinkhorn.pdf and I am curious about the properties of Sinkhorn distance and wondering what properties of the Sinkhorn ...
4
votes
1
answer
1k
views
What is the intuitive difference between Wasserstein-1 distance and Wasserstein-2 distance?
What is the intuitive difference between Wasserstein-1 distance and Wasserstein-2 distance, and how to know which one to use?
- 3,067
1
vote
1
answer
2k
views
What are the advantages of Wasserstein distance compared to Jensen-Shannon divergence?
What is the practical difference between Wasserstein metric and Jensen-Shannon divergence? Wasserstein metric is also referred to as Earth mover's distance.
From Wikipedia:
Wasserstein metric is a ...
- 3,067
2
votes
0
answers
70
views
How to Show Wasserstein Metric is Sum Invariant?
A paper I'm trying to understand states that that Wasserstein metric obeys certain properties, which I'd like to prove. This metric is defined for two random variables $U, V$ and $p \in (1, \infty)$ ...
- 699
7
votes
1
answer
1k
views
Multivariate Wasserstein metric for $n$-dimensions
I am a vegetation ecologist and poor student of computer science who recently learned of the Wasserstein metric. Application of this metric to 1d distributions I find fairly intuitive, and inspection ...
- 71
1
vote
1
answer
244
views
Optimal critic in Wasserstein GAN (WGAN)
In the Wasserstein GAN (generative adversarial network) paper they say that the critic needs to be optimal.
What does this mean practically?
They do more training iterations of critic (5) than the ...
- 33
1
vote
0
answers
363
views
Weight Clipping range with WGAN and relation with other factors
For a while, my code with WGAN has failed to generate quality images through a complicated multi-class database. My issue has been with implementation and not code.
Recently I read the WGAN-GP paper ...
0
votes
1
answer
99
views
Large gradient magnitude in WGAN-GP
I've implemented a minimal WGAN-GP on MNIST (code here).it kinda works and outputs some digits but loss/gradient magnitudes are so huge, for example:
$GradientPenalty \approx 10^{19}$
$D(\hat{x})- D(...
- 101
4
votes
1
answer
391
views
Upper Bound on the Wasserstein Distance
I'm interested to know if it's possible to construct an upper bound on the Wasserstein distance in terms of the Kolgomorov distance.
The Wasserstein distance can we written as
$$W_{1}\left(F, G\...
- 199
3
votes
1
answer
2k
views
Why is "weight clipping" needed for Wasserstein GANs?
I am reading the original paper on the Wasserstein GAN:
https://arxiv.org/pdf/1701.07875.pdf
and I came across this paragraph:
I don't understand the statement: "$\mathcal{W}$ is compact implies ...
- 141
1
vote
0
answers
49
views
Statistical estimators distance for close empirical distributions
Is it valid to argue that two empirical distributions $ p_1, p_2 $ having small Wasserstein distance $W_r(\cdot)$ for an order $ r $ will yield close MLE estimators for a statistical model ...
- 884
1
vote
0
answers
222
views
Wasserstein distance / EMD of two sets of 2D weighted points?
I am trying to implement a 2D version of the EMD/Wasserstein Distance to measure the distance of sets of 2D weighted points.
Let $A = \{a_{1}, a_{2}, ..., a_{m}\}$, be a weighted point set such that $...
- 180
8
votes
2
answers
2k
views
Relation Between Wasserstein Distance and KL-Divergence (Relative Entropy)
Consider the Wasserstein metric of order one $W_1$ (a.k.a. the Earth Movers Distance). I would like to know whether it is possible to link $W_1$ and Kullback–Leibler divergence (a.k.a. relative ...
- 81
7
votes
2
answers
7k
views
Calculate Earth Mover's Distance for two grayscale images
I am trying to calculate EMD (a.k.a. Wasserstein Distance) for these two grayscale (299x299) images/heatmaps:
Right now, I am calculating the histogram/distribution of both images. The histograms ...
- 115
7
votes
1
answer
2k
views
Earth Movers Distance and Maximum Mean Discrepency
By Kantorovich-Rubinstein duality the Earth Movers Distance (EMD)/Wasserstein Metric is equivalent to Maximum Mean Discrepancy (MMD) correct? See here for a more thorough explanation. Why then does ...
- 611
2
votes
1
answer
154
views
Prerequisites for Wasserstein GAN/Autoencoder
Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background?
The mentioned papers seem little tough if you don't have ...
- 5,546
3
votes
1
answer
70
views
Computing Wassertein Distance
For two probability measures $\mu$ and $\nu$, the Wassertein Distance is defined as $$W_p (\mu , \nu) = \left[ \inf\limits_{\gamma \in \Gamma} |x-y|^p \, d\gamma (x,y) \right] ^{\frac{1}{p}} \, , $$
...
- 205
1
vote
1
answer
319
views
Implementation of WAE-GAN does not match with the description in the paper
According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts.
in the algorithm ...
- 233
0
votes
0
answers
421
views
Difference between the Wasserstein metric, mallows metric and Earth mover's distance
I'm really confused, is there a difference between the Wasserstein metric, mallows metric and Earth mover's distance?
If yes What is it?
Thank you
- 457
4
votes
0
answers
987
views
Wasserstein distance between Gaussian and the empirical distribution
Wasserstein distance between two gaussians has a well known closed form solution. Does the same hold for the distance between a Gaussian with a fixed variance(say 1) and the empirical data ...
- 41
5
votes
2
answers
3k
views
Wasserstein Loss is very sensitive to model architecture
I am working on a class project where I compare the performance of GAN and WGAN. Since the only difference between GAN and WGAN is the Wasserstein loss, I chose one neural network model architecture ...
- 163
3
votes
0
answers
431
views
Distance or divergence for ordinal distribution
Measures like KL divergence can be symmetrized (into JS divergence). Bhattacharyya distance serves a similar function. Either is well-suited to both continuous distributions and discrete (e.g. ...
- 6,797
3
votes
0
answers
170
views
Prove the existence of a fixed point of a certain mapping of distributions
Let $\tilde{X}_0$ be some random variable on $\mathbb{R}^n$, with a strictly positive p.d.f..
Define:
$$X_0:=(\operatorname{var}{\tilde{X}_0})^{-\frac{1}{2}}(\tilde{X}_0-\mathbb{E}\tilde{X}_0),$$
...
- 475
4
votes
0
answers
610
views
In a WGAN, when is the generator's loss function ever used? [closed]
I've been building a Wasserstein GAN in Keras recently following the original Arjovsky implementation in PyTorch and ran across an issue I've yet to understand.
To my knowledge, the critic network is ...
- 41
2
votes
1
answer
914
views
Distance measure between discrete distributions (that contains 0) and uniform
I'm trying to choose a district metric that falls between 0 to 1 and lets me compare the distance between a uniform probability distribution and any given probability distribution (could be random, ...
65
votes
5
answers
25k
views
What is the advantages of Wasserstein metric compared to Kullback-Leibler divergence?
What is the practical difference between Wasserstein metric and Kullback-Leibler divergence? Wasserstein metric is also referred to as Earth mover's distance.
From Wikipedia:
Wasserstein (or ...
4
votes
2
answers
4k
views
Kullback-Leibler distance for comparing two distribution from sample points
I have two data samples of a value and I want to compute some distance which would represent the difference in their distribution.
I read about Kullback-Leibler distance which could be used for ...
- 61