5
$\begingroup$

Mahalanobis distance accounts for different variances of the marginal variables and correlations between the marginal variables. However, there is an implicit (maybe explicit) assumption that correlation is the right measure of dependence between the marginal variables. Certainly this need not be the case. We could be dealing with a very different type of copula than Gaussian, in which case, we can calculate Mahalanobis distance, but it isn't so meaningful.

Are there techniques for finding meaningful distances when the copula is not Gaussian? I'd really love it if the technique turned out to be Mahalanobis distance in the case of the Gaussian copula. An idea that I had involves the principal curvatures of the copula (eigenvalues of something?), though I haven't a clue what to do with that, and assuming differentiability becomes problematic when we deal with empirical distributions. (Maybe the estimate is a very different discussion.) Google Scholar seems not to return anything about doing differential geometry on the copula, however.

I think what I want to do is find confidence regions that need not be elliptical, though I am not 100% certain what I want to do with the distance measure.

$\endgroup$
3
  • 1
    $\begingroup$ Mahalanobis distance is perfectly meaningful when the first two moments exist. $\endgroup$ Commented Jun 20, 2019 at 16:47
  • $\begingroup$ Sure, we can calculate it if we can calculate the covariance matrix, but that doesn't make it a particularly useful measurement when the dependence is totally nonlinear. Imagine points tightly arranged in a circle. The center of that circle is extremely far from the mainstream of that distribution, which I would not expect Mahalanobis distance to catch. $\endgroup$
    – Dave
    Commented Jun 20, 2019 at 16:55
  • $\begingroup$ I mean a scatterplot with points tightly hugging the edge of a circle so that there is virtually no density in the center of the circle.. $\endgroup$
    – Dave
    Commented Jun 20, 2019 at 17:23

2 Answers 2

4
$\begingroup$

The Mahalanobis distance is a special case of the "null deviance"

Consider a random vector $\mathbf{X} \sim \text{N}(\boldsymbol{\mu}, \mathbf{\Sigma})$ with dimension $k$ and the Mahalanobis distance function:

$$D^2(\mathbf{x}) = (\mathbf{x} - \boldsymbol{\mu})^\text{T} \mathbf{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu}) \quad \quad \quad \text{for all } \mathbf{x} \in \mathbb{R}^k.$$

We can write the log-density for this distribution as:

$$\begin{align} \log p (\mathbf{x}|\boldsymbol{\mu}, \mathbf{\Sigma}) &= -\frac{k}{2} \log(2 \pi) - \frac{1}{2} \log \det \mathbf{\Sigma} - \frac{1}{2} (\mathbf{x} - \boldsymbol{\mu})^\text{T} \mathbf{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu}) \\[6pt] &= -\frac{k}{2} \log(2 \pi) - \frac{1}{2} \log \det \mathbf{\Sigma} - \frac{1}{2} D^2(\mathbf{x}) \\[6pt] &= \log p (\mathbf{x}|\mathbf{x}, \mathbf{\Sigma}) - \frac{1}{2} D^2(\mathbf{x}), \\[6pt] \end{align}$$

Using this result and setting $\hat{\boldsymbol{\mu}}_0 = \boldsymbol{\mu}$ and $\hat{\boldsymbol{\mu}}_\text{S} = \mathbf{x}$ gives the alternative expression:

$$\begin{align} D^2(\mathbf{x}) &= 2 \Big[ \log p (\mathbf{x}|\mathbf{x}, \mathbf{\Sigma}) - \log p (\mathbf{x}|\boldsymbol{\mu}, \mathbf{\Sigma}) \Big] \\[6pt] &= 2 \Big[ \log p (\mathbf{x}|\hat{\boldsymbol{\mu}}_\text{S}, \mathbf{\Sigma}) - \log p (\mathbf{x}|\hat{\boldsymbol{\mu}}_0, \mathbf{\Sigma}) \Big]. \\[6pt] \end{align}$$

Now, this latter expression is the "null deviance" that occurs when we compare a saturated model with a free mean parameter against the null model where the mean parameter is fixed (at the null value $\boldsymbol{\mu}$). Thus, the Mahalanobis distance can be considered to be a special case of the null deviance for this general comparison.

Since Mahalanobis distance is a special case of the null deviation, to extend the former concept you can simply use the latter. That is, to extend the concept to broader models and distributions, we need merely employ the null deviance for the same comparison (free mean versus fixed mean) in the broader model/distribution. If you are working with a parametric model then you can obtain the null deviance through the appropriate optimisation of the log-likelihood. If you are working non-parametrically, you will need to estimate the null deviance. In either case, the null deviance (or its estimate) constitutes a generalisation of the Mahalanobis distance that can be applied more broadly to models where the random vector is not normally distributed.

$\endgroup$
2
  • $\begingroup$ How would I apply this empirically, say to data forming a circle-looking scatter plot? set.seed(2021); N <- 1000; a <- -1; b <- 1; x <- seq(a, b, (b - a)/(N - 1)); y <- sqrt(1 - x^2); x <- c(x, x) + rnorm(2*N, 0, 0.01); y <- c(y, -y) + rnorm(2*N, 0, 0.01) From the graph, I can tell that $(0, 0)$ is *really) far away from the rest of the points, but how far is far? How much further is it from the rest of the points than $(1, 1)$ is? $\endgroup$
    – Dave
    Commented Jun 24, 2021 at 12:22
  • $\begingroup$ If you have an underlying parametric distribution of known form then you would use that to compute null deviance. If you are working non-parametrically then I guess you would use some kind of non-parametric estimator for the density function to get an estimate null deviance. I'm not sure of further specifics. $\endgroup$
    – Ben
    Commented Jun 25, 2021 at 22:20
1
$\begingroup$

The implicit (maybe explicit) assumption that correlation is the right measure of dependence in the case of the Gaussian copula (with normal magins, or else "normalized" margins, see e.g. the nonparanormal) is very much explicit. Indeed, the Mahalanobis distance corresponds to the log-likelihood of the data in this case.

So say you have two iid random vectors ${\bf X},{\bf Y} \sim N({\bf \mu},{\bf{\Sigma}})$, their Mahalanobis distance is $$ {\rm Maha}({\bf X},{\bf Y}|{\bf S}) = ({\bf X}-{\bf Y})^{\top}{\bf S^{-1}}({\bf X}-{\bf Y}), $$ where ${\bf S}$ is an estimate of ${\bf \Sigma}$ (if not known). Interpreting ${\bf Y}$ as an estimator of ${\bf \mu}$ (an unbiased one in fact), this is the log-likelihood of ${\bf X}$ (negated and up to some factor that we can neglect). Note that one can judge that it is more appropriate, in this case, to use an estimate ${\bf S}$ of $\mathbb{Var}({\bf X}-{\bf Y})=2{\bf \Sigma}$, i.e. to evaluate the log-likelihood of ${\bf X}-{\bf Y} \sim N({\bf 0},2{\bf \Sigma})$, but this turns out to be an equivalent distance.

Now, to answer your question, I think that what your are looking for is some similar meaning in the case of a different copula. A good first step might be to (to some extent) explicitly write your likelihood function, or that of $({\bf X}-{\bf Y})$, and look for that meaning.

And if you do not want to depend on distributional assumptions, then I guess you better begin by answering the question: "what distance do I find meaningful for my application?" rather than deriving a meaningful distance for a given distribution. As pointed out by steveo'america, the Mahalanobis distance (assuming that your data has finite second moment) might actually be what you are looking for...

Hope that helps.

$\endgroup$
1
  • 1
    $\begingroup$ I think I want something more flexible than Mahalanobis distance so that I would get a better confidence region when the copula is not Gaussian, but when the copula is Gaussian, it gives the Mahalanobis distance, kind of like how we might use median as a location estimator for skewed distributions, but it equals the mean when we have a normal distribution (something like that). $\endgroup$
    – Dave
    Commented Jul 24, 2019 at 13:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.