Questions tagged [wasserstein]

The Wasserstein metric or Earth Movers Distance is a distance function between probability distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Can we train normalizing flows with Wasserstein distance?

To train flow based models, you usually either use forward or reverse kl as your loss function. My question is, can you use wasserstein distance directly as your loss function to replace kl? I have ...
user avatar
  • 101
0 votes
0 answers
7 views

Problem for training Wasserstein GAN

I'm trying to train a Wasserstein GAN to guess sparse one-hot encoded matrices (0/1), in particular I've reimplemented the same architecture proposed in this paper. The problem, as you can see, is ...
user avatar
0 votes
0 answers
11 views

Wasserstein Adversarial Imitation Learning

in the paper, its mentioned that the authors used L2 regularization during training as shown below. −1/4ε(r(y) − r(x) − d(x, y))2 but it is not clear to me how to implement it, any hints?.
user avatar
  • 1
0 votes
0 answers
10 views

Getting rid of the additive degree of freedom for discriminators of WGAN-GP's

Setting: Discriminators in WGAN-GP's are trained to minimise the following loss functional over functions D: Here I have been playing around with training a critic (simple convolutional network ...
user avatar
0 votes
0 answers
21 views

Estimating empirically the Wasserstein distance

I have a dataset of the form $\{x_i,y_i,y_i'\}$, where $y_i\sim p(\cdot|x_i)$ and $y_i'\sim q(\cdot|x_i)$, while $x_i$ itself has a distribution $d(\cdot)$ Is there I way to estimate $$\mathbb E_{x\...
user avatar
0 votes
0 answers
48 views

I need help understanding the meaning of the loss values of a WGAN with Gradient Penalty

I am currently working on training a Auxiliary Classifier Wasserstein GAN with Gradient Penalty. I based my implementation off of https://keras.io/examples/generative/wgan_gp/ (to which I added the ...
user avatar
1 vote
1 answer
291 views

Wasserstein distance and Kolmogorov-Smirnov statistic as measures of effect size

So I've been dealing with 2 sample hypothesis testing with very large samples (around 20,000s each). Whenever I test for the equality of distribution I always reject the null hypothesis, even though ...
user avatar
  • 23
0 votes
0 answers
25 views

Measure for evaluating a density estimation procedure

Given an implementation of a multivariate density estimation scheme, what would be a suitable measure to evaluate the accuracy of the procedure? I am currently evaluating the procedure using three ...
user avatar
  • 964
0 votes
0 answers
141 views

Interpretation of upper bound on the Wasserstein Distance

I am trying to interpret the 2-Wasserstein distance and the upper bound on it. Let's say I have 2-Wasserstein distance between two distributions to be $x$, and I have an upper bound on it which gives ...
user avatar
  • 185
1 vote
1 answer
335 views

Intuition on Wasserstein Distance

I've been trying to familiarize myself with the Wasserstein distance and saw this answer on StackExchange by @antike that at first made a lot of sense, but then it didn't (to me, of course). In the ...
user avatar
  • 163
0 votes
1 answer
77 views

How can one compute the "average" of a dataset of histograms that minimizes the mean Earth Mover's Distance between all data points and average?

It is my understanding that when the distance metric is euclidean distance, the mean of a dataset minimizes the average distance between all data points and the computed "mean". In the case ...
user avatar
1 vote
0 answers
35 views

What is this probability distribution?

Thank you in advance for any suggestions or feedback. I have a discrete 1D probability distribution represented as a vector $\textbf{p}$, $p_i = p(x_i)$. I am interested in finding the Wasserstein (...
user avatar
1 vote
1 answer
695 views

When is my Wasserstein GAN-GP overfitting?

I have a hard time interpreting the WGAN-GP losses attached. At which epoch is D and/or G overfitting? The quality improves a lot overtime, yet the generator loss at later epochs does not appear on ...
user avatar
0 votes
0 answers
286 views

Wasserstein distance for categorical data? Relationship to TVD?

Is the Wasserstein distance applicable for categorical data? e.g. if we have the distribution of different coloured balls in two bags, ...
user avatar
1 vote
0 answers
71 views

Reference for Wasserstein distance for generalized exponential family and generalized extreme value probability distributions

The Wikipedia page gives the analytical results for computing the Wasserstein metric between two multivariate normal distributions, and I thought it used to give results for several other commonly ...
user avatar
  • 297
-1 votes
1 answer
47 views

K-Lipschitz for the Wasserstein GANs

I am trying to follow this blog for Wasserstein loss for Generative Adversarial Networks: From GAN to WGAN. Actually, I am trying to follow the logic behind the K-Lipschitz continuity. This post in ...
user avatar
1 vote
1 answer
17 views

How to check whether a partitioning is more balanced?

For example consider the following two partitions that add approximately to 100 percent. Partition 1 : 41% 36% 14% 5% 2% 1% <1% <1% Partition 2 : 82% 10% 4% 3% <1% <1% <1% <1% ...
user avatar
0 votes
0 answers
43 views

Horizontal vs Vertical optimal transport of probability distributions

Optimal transport has a primal and dual form. My question is: Is the primal formulation of OT intended for horizontal movement of probability mass, whereas the dual formulation is more geared towards ...
user avatar
  • 3,067
0 votes
0 answers
111 views

Does Wasserstein distance require the source and target distributions to have the same mass?

If we minimize the Wasserstein loss, $$W_1 (P_S, P_T) = \underset{\gamma \in \Pi}{\text{min }} \sum_{x,y} |x-y|\gamma(x,y)$$ which means we are looking for the coupling $\gamma(x,y)$ that minimizes ...
user avatar
  • 3,067
0 votes
0 answers
38 views

If entropy is the underlying measure for KL-divergence, what is the underlying measure for the Wasserstein distance?

If entropy is the basis measure underlying KL-divergence (aka relative entropy), what is the basis measure underlying the Wasserstein distance?
user avatar
  • 3,067
1 vote
0 answers
77 views

Properties of Sinkhorn distance

I am reading the paper by Cuturi http://www.marcocuturi.net/Papers/cuturi13sinkhorn.pdf and I am curious about the properties of Sinkhorn distance and wondering what properties of the Sinkhorn ...
user avatar
4 votes
1 answer
1k views

What is the intuitive difference between Wasserstein-1 distance and Wasserstein-2 distance?

What is the intuitive difference between Wasserstein-1 distance and Wasserstein-2 distance, and how to know which one to use?
user avatar
  • 3,067
1 vote
1 answer
2k views

What are the advantages of Wasserstein distance compared to Jensen-Shannon divergence?

What is the practical difference between Wasserstein metric and Jensen-Shannon divergence? Wasserstein metric is also referred to as Earth mover's distance. From Wikipedia: Wasserstein metric is a ...
user avatar
  • 3,067
2 votes
0 answers
70 views

How to Show Wasserstein Metric is Sum Invariant?

A paper I'm trying to understand states that that Wasserstein metric obeys certain properties, which I'd like to prove. This metric is defined for two random variables $U, V$ and $p \in (1, \infty)$ ...
user avatar
7 votes
1 answer
1k views

Multivariate Wasserstein metric for $n$-dimensions

I am a vegetation ecologist and poor student of computer science who recently learned of the Wasserstein metric. Application of this metric to 1d distributions I find fairly intuitive, and inspection ...
user avatar
1 vote
1 answer
244 views

Optimal critic in Wasserstein GAN (WGAN)

In the Wasserstein GAN (generative adversarial network) paper they say that the critic needs to be optimal. What does this mean practically? They do more training iterations of critic (5) than the ...
user avatar
  • 33
1 vote
0 answers
363 views

Weight Clipping range with WGAN and relation with other factors

For a while, my code with WGAN has failed to generate quality images through a complicated multi-class database. My issue has been with implementation and not code. Recently I read the WGAN-GP paper ...
user avatar
0 votes
1 answer
99 views

Large gradient magnitude in WGAN-GP

I've implemented a minimal WGAN-GP on MNIST (code here).it kinda works and outputs some digits but loss/gradient magnitudes are so huge, for example: $GradientPenalty \approx 10^{19}$ $D(\hat{x})- D(...
user avatar
4 votes
1 answer
391 views

Upper Bound on the Wasserstein Distance

I'm interested to know if it's possible to construct an upper bound on the Wasserstein distance in terms of the Kolgomorov distance. The Wasserstein distance can we written as $$W_{1}\left(F, G\...
user avatar
  • 199
3 votes
1 answer
2k views

Why is "weight clipping" needed for Wasserstein GANs?

I am reading the original paper on the Wasserstein GAN: https://arxiv.org/pdf/1701.07875.pdf and I came across this paragraph: I don't understand the statement: "$\mathcal{W}$ is compact implies ...
user avatar
1 vote
0 answers
49 views

Statistical estimators distance for close empirical distributions

Is it valid to argue that two empirical distributions $ p_1, p_2 $ having small Wasserstein distance $W_r(\cdot)$ for an order $ r $ will yield close MLE estimators for a statistical model ...
user avatar
  • 884
1 vote
0 answers
222 views

Wasserstein distance / EMD of two sets of 2D weighted points?

I am trying to implement a 2D version of the EMD/Wasserstein Distance to measure the distance of sets of 2D weighted points. Let $A = \{a_{1}, a_{2}, ..., a_{m}\}$, be a weighted point set such that $...
user avatar
8 votes
2 answers
2k views

Relation Between Wasserstein Distance and KL-Divergence (Relative Entropy)

Consider the Wasserstein metric of order one $W_1$ (a.k.a. the Earth Movers Distance). I would like to know whether it is possible to link $W_1$ and Kullback–Leibler divergence (a.k.a. relative ...
user avatar
  • 81
7 votes
2 answers
7k views

Calculate Earth Mover's Distance for two grayscale images

I am trying to calculate EMD (a.k.a. Wasserstein Distance) for these two grayscale (299x299) images/heatmaps: Right now, I am calculating the histogram/distribution of both images. The histograms ...
user avatar
  • 115
7 votes
1 answer
2k views

Earth Movers Distance and Maximum Mean Discrepency

By Kantorovich-Rubinstein duality the Earth Movers Distance (EMD)/Wasserstein Metric is equivalent to Maximum Mean Discrepancy (MMD) correct? See here for a more thorough explanation. Why then does ...
user avatar
  • 611
2 votes
1 answer
154 views

Prerequisites for Wasserstein GAN/Autoencoder

Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background? The mentioned papers seem little tough if you don't have ...
user avatar
3 votes
1 answer
70 views

Computing Wassertein Distance

For two probability measures $\mu$ and $\nu$, the Wassertein Distance is defined as $$W_p (\mu , \nu) = \left[ \inf\limits_{\gamma \in \Gamma} |x-y|^p \, d\gamma (x,y) \right] ^{\frac{1}{p}} \, , $$ ...
user avatar
1 vote
1 answer
319 views

Implementation of WAE-GAN does not match with the description in the paper

According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts. in the algorithm ...
user avatar
0 votes
0 answers
421 views

Difference between the Wasserstein metric, mallows metric and Earth mover's distance

I'm really confused, is there a difference between the Wasserstein metric, mallows metric and Earth mover's distance? If yes What is it? Thank you
user avatar
  • 457
4 votes
0 answers
987 views

Wasserstein distance between Gaussian and the empirical distribution

Wasserstein distance between two gaussians has a well known closed form solution. Does the same hold for the distance between a Gaussian with a fixed variance(say 1) and the empirical data ...
user avatar
5 votes
2 answers
3k views

Wasserstein Loss is very sensitive to model architecture

I am working on a class project where I compare the performance of GAN and WGAN. Since the only difference between GAN and WGAN is the Wasserstein loss, I chose one neural network model architecture ...
user avatar
  • 163
3 votes
0 answers
431 views

Distance or divergence for ordinal distribution

Measures like KL divergence can be symmetrized (into JS divergence). Bhattacharyya distance serves a similar function. Either is well-suited to both continuous distributions and discrete (e.g. ...
user avatar
3 votes
0 answers
170 views

Prove the existence of a fixed point of a certain mapping of distributions

Let $\tilde{X}_0$ be some random variable on $\mathbb{R}^n$, with a strictly positive p.d.f.. Define: $$X_0:=(\operatorname{var}{\tilde{X}_0})^{-\frac{1}{2}}(\tilde{X}_0-\mathbb{E}\tilde{X}_0),$$ ...
user avatar
  • 475
4 votes
0 answers
610 views

In a WGAN, when is the generator's loss function ever used? [closed]

I've been building a Wasserstein GAN in Keras recently following the original Arjovsky implementation in PyTorch and ran across an issue I've yet to understand. To my knowledge, the critic network is ...
user avatar
2 votes
1 answer
914 views

Distance measure between discrete distributions (that contains 0) and uniform

I'm trying to choose a district metric that falls between 0 to 1 and lets me compare the distance between a uniform probability distribution and any given probability distribution (could be random, ...
user avatar
65 votes
5 answers
25k views

What is the advantages of Wasserstein metric compared to Kullback-Leibler divergence?

What is the practical difference between Wasserstein metric and Kullback-Leibler divergence? Wasserstein metric is also referred to as Earth mover's distance. From Wikipedia: Wasserstein (or ...
user avatar
4 votes
2 answers
4k views

Kullback-Leibler distance for comparing two distribution from sample points

I have two data samples of a value and I want to compute some distance which would represent the difference in their distribution. I read about Kullback-Leibler distance which could be used for ...
user avatar