# Comparing two distributions in Fourier space

There exist a number of tools that provide a distance between two continuous probability distributions. Most (semi)distances, like the Kullback-Leibler divergence, use probability density functions. However, the literature is quite sparse when it comes to comparing two distributions in Fourier space, i.e. via their characteristic function. Is there an elegant way to do so?

• I thought of using spectral coherence (en.wikipedia.org/wiki/Coherence_%28signal_processing%29) but do not know how to apply it to this case Commented May 6, 2014 at 17:33
• Did you find an answer to this?
– a06e
Commented Apr 5, 2018 at 18:25
• Google for "empirical characteristic function", it can be used for construction hypothesis tests, and certainly for constructing distances. Commented Apr 8, 2018 at 15:09

One notable distance that can be considered in Fourier space is the maximum mean discrepancy (MMD). One first selects a positive semi-definite kernel $$k : \mathcal X \times \mathcal X \to \mathbb R$$ corresponding to a reproducing kernel Hilbert space (RKHS) $$\mathcal H_k$$. The MMD is then

\begin{align} \operatorname{MMD}_k(\mathbb P, \mathbb Q) &= \sup_{f \in \mathcal H_k : \lVert f \rVert_{\mathcal H_k} \le 1} \mathbb E_{X \sim \mathbb P}[ f(X)] - \mathbb E_{Y \sim \mathbb Q}[ f(Y)] \\&= \left\lVert \mathbb E_{X \sim \mathbb P}[k(X, \cdot)] - \mathbb E_{Y \sim \mathbb Q}[k(Y, \cdot)] \right\rVert \\&= \sqrt{ \mathbb{E}_{\substack{X, X' \sim \mathbb P\\Y, Y' \sim \mathbb Q}}\left[ k(X, X') + k(Y, Y') - 2 k(X, Y) \right] } .\end{align} You might be familiar with the energy distance; that is a special case of the MMD for a particular choice of kernel. This is a proper metric for many choices of kernel, called characteristic kernels; it is always a semimetric.

What does this have to do with the Fourier transform? Well, if $$\mathcal X = \mathbb R^d$$ and $$k(x, y) = \psi(x - y)$$, so that $$\psi : \mathbb R^d \to \mathbb R$$ is a positive-definite function, then it turns out the MMD can also be written as $$\operatorname{MMD}_k(\mathbb P, \mathbb Q) = \sqrt{\int \left\lvert \varphi_{\mathbb P}(\omega) - \varphi_{\mathbb Q}(\omega) \right\rvert^2 \, \mathrm{d}\hat\psi(\omega)}$$ where $$\varphi$$ denotes the characteristic function, and $$\hat\psi$$ is the Fourier transform of $$\psi$$ in the measure sense. (It will always be a finite nonnegative measure; you can see from this definition that a translation-invariant kernel is characteristic iff its Fourier transform is everywhere positive.) For a proof, see Corollary 4 of

Sriperumbudur et al., Hilbert space embeddings and metrics on probability measures, JMLR 2010.

The MMD – which you can easily estimate via the third form above – thus compares distributions by the $$L_2$$ distance between their full characteristic functions, with frequencies weighted according to the choice of kernel. For example, the common Gaussian kernel $$k(x, y) = \exp\left( -\frac{1}{2\sigma^2} \lVert x - y \rVert^2 \right)$$ will weight the frequencies with a Gaussian with mean 0 and variance $$1/\sigma^2$$.

Sometimes it's better, and sometimes computationally faster / more informative, to instead compare the characteristic functions at particular locations rather than everywhere. It turns out it's better to make a slight tweak, evaluating differences in smoothed characteristic functions at random locations:

Chwialkowski et al., Fast Two-Sample Testing with Analytic Representations of Probability Measures, NeurIPS 2015.

A follow-up work finds the most informative frequencies to test, rather than random:

Jitkrittum et al., Interpretable Distribution Features with Maximum Testing Power, NeurIPS 2016.

These are all closely connected to classical tests based on empircal characteristic functions as mentioned by kjetil in the comments.

• Great reply to a question I asked almost 6 years ago! Commented Jan 10, 2020 at 11:37
• @yannick It popped up in the related bar, figured I could share for posterity. :) Commented Jan 10, 2020 at 17:18