Relations between the energy distance and MMD

Question

I was wondering if there's any relation between the two metrics. Both measure the distance between distributions (or samples of them). And they seem quite similar.

The energy distance can be summarized as twice the mean of the between sample distances minus the means of the within sample distances of each sample:

$$ED(\theta_1,\theta_2) = 2\mathbb E[\Vert\theta_1-\theta_2\Vert] - \mathbb E[\Vert \theta_1-\theta_1'\Vert] - \mathbb E[\Vert\theta_2-\theta_2'\Vert]$$

The MMD with a Gaussian kernel seems to be very related to it: the negative squared-distances are exponentiated with scale, then the opposite sum is taken: the within's minus twice the between:

$$ MMD(x,y) = \mathbb E\left[e^{-\Vert x-x'\Vert^2/c}\right] + \mathbb E\left[e^{-\Vert y-y'\Vert^2/c}\right] -2\mathbb E\left[e^{-\Vert x-y\Vert^2/c}\right] $$

Answer 1

Disclosure: I'm not an expert in the topic, so if someone knows the subject better, please correct me where I'm wrong!

You are correct - they are related. But to be clear the definitions you have provided are of the squared ED and squared MMD.

MMD (and hence squared MMD) is a general metric that measures distance between probability distributions, is non-negative, and is zero only if both distributions are the same. The squared MMD can be written as

$$ \mathrm{MMD}^2(q, p) = \mathbb{E}_{x, \tilde x \sim p} \left[ k(x, \tilde x) \right] + \mathbb{E}_{y, \tilde y \sim q} \left[ k(y, \tilde y) \right] - 2 \mathbb{E}_{x \sim p, y \sim q} \left[ k(x, y) \right], $$ where $k(\cdot, \cdot)$ is the characteristic kernel function.

$\mathrm{MMD}^2$ is particularly attractive because it can be approximated using only samples from the two distributions, making it very useful for assessing implicit distributions (distributions that do not have a computationally tractable form of their pdfs/pmfs).

Different characteristic kernels $k(\cdot, \cdot)$ can produce metrics with different behaviors. A popular choice is the Gaussian kernel as you have mentioned.

Using another popular kernel, the Euclidean distance, in $\mathrm{MMD}^2$ you can recover $\mathrm{ED}^2$, and hence ED inherits the properties of MMD (non-negativity, and uniqueness).