4
$\begingroup$

When we study correlation or association in real data, do we always (implicitly) assume a finite second order moment for any hypothetical population distribution? If we do not assume this, what statistical measure may we use?

I am not sure if it makes sense to study correlation (or association in general) without assuming a finite second order moment for a generating probabilistic model. In the generalized version of pearson correlation (wikipedia general correlation), the norms in the denominator of the expression should implicitly assume a finite second order moment.

I ask this question simply to keep track of what we do and do not assume when we look into association in datasets.

I think that association does not make sense without assuming a finite second order moment for any population distribution. However, maybe there is a nonparametric measure that I am not familiar with. Since both Spearman's and Kendall's correlation coefficients can be expressed in the form in 1, I think this implicit assumption of a finite second order moment applies to these two association measures also. Any insights would be greatly appreciated!

This stackexchange question seems relavent: PCA without finite second moment. However, as far as I can tell, it does not definitively answer the question as I understand it.

$\endgroup$
3
  • 3
    $\begingroup$ Spearman does not even assume the existence of a first moment; you can calculate the Spearman correlation between two Cauchy variates, for example, and it's perfectly valid. Similarly for Kendall. $\endgroup$ – jbowman Feb 24 '20 at 17:11
  • 1
    $\begingroup$ Don't forget about the copula! Remember the theorem that $X\sim f_X(x)$ and $Y\sim g_Y(y)$ are independent if and only if $h_{X,Y}(x,y) = f_X(x)g_Y(y)$? Well if $X$ and $Y$ are not independent, then there's some $c(x,y)$ such that $h_{X,Y}(x,y) = f_X(x)g_Y(y)c(x,y)$, and $c(x,y)$ is called the copula density. (The "copula" is the CDF of $c$, and we typically write both as functions of the CDFs $F_X(x)$ and $G_Y(y)$.) (Also, this might require continuous variables to work out this cleanly.) $\endgroup$ – Dave Feb 24 '20 at 17:55
  • $\begingroup$ I definitely need to read more about copulas, this is definitely something I had in the back of my mind when I was thinking of this question. Thanks for the pointer! $\endgroup$ – Sleepy 17 Feb 24 '20 at 17:57
1
$\begingroup$

Consider two i.i.d. standard Cauchy variates $X$ and $Y$.

Obviously the Pearson correlation between $X$ and $Y$, although it can be calculated on the basis of a sample, is not meaningful in this case. The Spearman and Kendall correlations, however, don't make any assumptions about existence of moments, and, as such, are perfectly valid measures of correlation (given what they actually indicate, e.g., a monotonic relationship in the case of Spearman.)

For demonstration purposes, here's some histograms of 10,000 calculations of the three correlations between 100 draws of $X$ and $Y$. First the code, written for clarity rather than speed, then the plots:

library(data.table)

rho <- data.table(pearson=rep(0,10000), spearman=rep(0,10000), kendall=rep(0,10000))

for (i in 1:nrow(rho)) {
   x <- rcauchy(100)
   y <- rcauchy(100)
   rho[i, ':='(pearson = cor(x,y,method="pearson"),
               spearman = cor(x,y,method="spearman"),
               kendall = cor(x,y,method="kendall"))]
}

enter image description here

Not to be relied upon, I think you'll agree. The story is different for Spearman and Kendall: enter image description here

enter image description here

In both these cases, the histograms seem quite reasonable given the sample size of 100. Of course, the math is what really counts, and in both these cases, there are no distributional assumptions involving moments whatsoever.

The reason why Spearman in particular can get away with this is that it operates on data that has been monotonically transformed onto the set of integers from $1$ to $N$, where $N$ is the sample size. This transformed data has moments of all orders, regardless of what the characteristics of the original data may have been. Kendall goes even further; it only cares about the relative order of the ranks of the pairs $(x_i, y_i)$ and $(x_j, y_j)$, not the values of the ranks themselves. In effect, it transforms the data onto $\{-1,1\}$. In both cases, the underlying distributions are irrelevant, except for the exclusion of the possibility of ties.

$\endgroup$
4
  • $\begingroup$ Thank you very much for this detailed answer! I think I understand my misconception of these two nonparametric measures of correlation. The sample versions of the correlations incorporate the norms of appropriate statistics. But they do not assume a second moment of the population. $\endgroup$ – Sleepy 17 Feb 24 '20 at 17:51
  • $\begingroup$ This might be a bad question, but is there such thing as a nonparametric generative model? I always see generative models that are parameterized, e.g., something in the exponential family. $\endgroup$ – Sleepy 17 Feb 24 '20 at 17:53
  • $\begingroup$ That's worthy of a whole new question! $\endgroup$ – jbowman Feb 24 '20 at 17:54
  • $\begingroup$ I tried to make a more general question out of that comment: stats.stackexchange.com/questions/451150/… $\endgroup$ – Sleepy 17 Feb 24 '20 at 19:54
1
$\begingroup$

I am not sure if it is what you are looking for, but there is a beautiful literature that studies relation between stochastic variables without the second moment:

On a new measure of covariation among stable variables

Correlation in $L^p$ spaces

$\endgroup$
1
  • $\begingroup$ Thank you for pointing me to this literature, I will definitely take a look. $\endgroup$ – Sleepy 17 Feb 24 '20 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.