1
$\begingroup$

I have two heavy-tailed random variables and want to know if they are correlated. While I only have estimates for the tail exponent, it is in a ballpark so that variance would normally not exist. The mean might exist but may be slow to converge.

I understand that using the Pearson correlation coefficient or a Least Squares regression are bad ideas as they rely on variance and covariance.

I found a paper by Balkema and Embrechts that considers regressions for heavy-tailed variables and heavy-tailed errors and concludes that for my case (non-existing variance), least squares and also least absolute deviation perform poorly. The authors suggest various more exotic loss functions for which they show the regression to perform better in that case. They incidentally also mention that least squares will be fine if the errors have finite variance. (I do not know whether they do or how to find out without performing the regression first.)

What is the best practice for this case?

I seem to remember seeing Spearman rank correlation being used. While this would certainly get rid of extreme observations (which cause infinite moments in heavy-tailed data), it also seems to blow up minor differences in observations with similar values (which would be expected in the body of the distribution for heavy-tailed distributions).

The solutions proposed by Balkema and Embrechts are probably not implemented in R or python and I am reluctant to implement this myself as it creates an additional source of errors.

Improve this question
$\endgroup$
6
  • $\begingroup$ Distance correlations are an unrestricted approach to pairwise dependence. $\endgroup$
    – user78229
    Commented Nov 7, 2019 at 18:21
  • $\begingroup$ @MikeHunter Thanks for the suggestion of distance correlations. It is not clear to me why they would generally work better with heavy-tailed data. Do you have some intuition or proof or reference where I can read about it? $\endgroup$
    – 0range
    Commented Nov 7, 2019 at 21:14
  • $\begingroup$ Check this reference for a comparison of several measures of dependence including distance correlations. Clark, M. A. Comparison of Correlation Measures, htps://m-clark.github.io/doc/CorrelationComparison.pdf Clark discusses the limitations of Kendall's tau. $\endgroup$
    – user78229
    Commented Nov 14, 2019 at 19:00
  • $\begingroup$ @MikeHunter Thanks; this document is very interesting. I assume you meant this link and I assume you meant that it discusses distance correlations, not Kentall's $\tau$ (which is not referenced). Unfortunately, the document does not address the problem of heavy tails (which is a completely different issue from non-linearity). It does, however, include a discussion of Mutual Information Content which takes an entropy-based approach and seems a bit more robust to heavy tails (although that is just a feeling; I cannot prove it). $\endgroup$
    – 0range
    Commented Nov 14, 2019 at 19:28
  • $\begingroup$ An explicit discussion of heavy-tails isn't needed wrt distance correlations, at least in my opinion, since they are capturing dependence of any relationship. Note, however, that the value returned says nothing about shape or direction. It only estimates the magnitude of the association. As the author of the article notes, DCs are the best all-purpose dependence metric. $\endgroup$
    – user78229
    Commented Nov 20, 2019 at 15:12

1 Answer 1

Reset to default
1
$\begingroup$

Try to use the Kendall's Tau correlation. Kendall's Tau (Kendall's Rank Correlation Coefficient) is a measure of nonlinear dependence between two random variables.

https://stanfordphd.com/KendallsTau.html

Hope this helps.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks for suggesting Kendall's $\tau$. Note that the problem with heavy-tailed data is not so much non-linearity, but non-convergence of moments. There seems to be some evidence (e.g. this paper) that Kendall's $\tau$ performs better than Pearson's R for heavy-tailed data (as does Spearman's $\rho$ according to the paper), but I'd like to understand why and whether this is universally the case. $\endgroup$
    – 0range
    Commented Nov 7, 2019 at 21:10
  • $\begingroup$ Hi @Orange, the reason is, if I am not wrong, the fundamental assumptions of the correlation--normality. The date should be normal-ish distributed, which suggests they shouldn't be skewed. Kendall's Tau's assumption is less stringent than the other since it uses a ranking as part of the calculation. Hope this helps. $\endgroup$
    – Bill Chen
    Commented Nov 7, 2019 at 21:14
  • $\begingroup$ Thanks @BillChen - So if I understand you correctly, Pearson's R and Least Squares etc etc come with restrictive assumptions (normality of the coefficient estimates, non-heavy-tailedness albeit not necessarily normality of errors?), while any rank-based correlation coefficients do not? $\endgroup$
    – 0range
    Commented Nov 7, 2019 at 22:01
  • $\begingroup$ Roughly correct. "Kendall's tau correlation coefficient (Kendall's tau-b, for short) is a nonparametric measure of the strength and direction of association that exists between two variables measured on at least an ordinal scale. It is considered a nonparametric alternative to the Pearson’s product-moment correlation when your data has failed one or more of the assumptions of this test. It is also considered an alternative to the nonparametric Spearman rank-order correlation coefficient (especially when you have a small sample size with many tied ranks)." $\endgroup$
    – Bill Chen
    Commented Nov 7, 2019 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.