1
$\begingroup$

I am dealing in a data science project with correlation analyses using pearson and distance correlation.

While trying to understand the differences between them, I learned about the differences by reading Szekely's paper and finding this very good question here.

So in my understanding the main difference of both is the methodological approach. I understand the underlying covariance of pearson's correlation is influenced by the sum of distances to a centroid (the mean), while for the distance covariance it is the sum of distances to all points of the data. The covariances in both cases are defined as the expectation value of those distances. The math approach to calculate all following quantities like correlation etc is equal for both.

The only difference is the definition of the covariance

  • "usual" covariance: ${\displaystyle \operatorname {cov} (X,Y):=\operatorname {E} {{\big [}(X-\operatorname {E} [X])(Y-\operatorname {E} [Y]){\big ]}}}$: Linear definition
  • Distance covariance: ${\displaystyle \operatorname {dCov} ^{2}(X,Y):=\operatorname {E} {\big [}d_{\mu }(X,X')d_{\nu }(Y,Y'){\big ]}}$: Squared definition.

Why is this the case?

$\endgroup$
10
  • 2
    $\begingroup$ If $X$ and $Y$ are vectors, then the "usual" covariance is in fact a matrix $E\left[ (X - E[X]) (Y - E[Y])^\top \right]$, while the distance covariance is a scalar. $\endgroup$
    – Frank
    Sep 24 '18 at 13:35
  • $\begingroup$ (This isn't the most fundamental difference, of course.) $\endgroup$
    – Frank
    Sep 24 '18 at 13:41
  • $\begingroup$ Thanks for your comment. The main difference in constructing the two methods is clear. I am specifically interested in why the one covariance is defined squared, while the other is not. Any idea? $\endgroup$ Sep 25 '18 at 9:02
  • 1
    $\begingroup$ Suppose $X$ and $Y$ are scalars. Their covariance can be negative, so a definition like $\operatorname{cov}(X,Y)^2 = E[ \dots ]^2$ would be ambiguous. But their distance covariance is nonnegative so there is no ambiguity. $\endgroup$
    – Frank
    Sep 25 '18 at 13:49
  • 1
    $\begingroup$ Anyway, the notation dCov² comes from the fact that this quantity is a squared distance between functions, $\|f_{X,Y} - f_X f_Y\|^2_w$, as explained on pages 3 and 4 of the Szekely paper. $\endgroup$
    – Frank
    Sep 25 '18 at 13:54
2
+50
$\begingroup$

The key here is in the exposition on pages 3 and 4 of Szekely's paper. The distance covariance is defined as the distance between two functions: it's a scalar, it must be nonnegative, and it's only zero if the two functions are the same.

In this case, let $X$ and $Y$ be (possibly dependent, possibly vector-valued) random variables, let $f_{X,Y}$ be the characteristic function of their joint distribution, and let $f_X$ and $f_Y$ be the characteristic functions of their marginal distributions. If $X$ and $Y$ are independent then $f_{X,Y} = f_X f_Y$, so we can measure "how dependent" they are using a distance between these functions. Define the squared distance covariance to be the squared distance between these functions: $$ \operatorname{dCov}^2(X,Y) := \|f_{X,Y} - f_X f_Y \|^2_w $$ where the exact definition of $\| \cdot \|^2_w$ is described on pages 4 and 5. This is defined as $\operatorname{dCov}^2$ to emphasize that it's a squared distance between functions.

Why isn't the covariance defined similarly? The covariance isn't a distance. For vector $X$ and $Y$ it's a matrix, not a scalar; for scalar $X$ and $Y$ it can be negative. If we defined squared covariance as $\operatorname{Cov}(X,Y)^2 = E \left[ (X - E[X]) (Y - E[Y]) \right]^2$ it would still be ambiguous what $\operatorname{Cov}(X,Y)$ would be: is it positive or is it negative?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.