Given two vectors $\mathbb{x},\mathbb{y} \in \mathbb{R}^n$ let $r_p$ be the Pearson correlation between the two vectors and let $r_k$ be the Kendall correlation. You can assume that $x,y$ follow a bivariate normal distribution.

I want to show empirically how sensitive Pearson and Kendall correlations are to outliers by demonstrating how much they both change when increasingly distant outliers are added to $x,y$.

That is, if $\hat{r_p}$ and $\hat{r_k}$ are the correlations of $x,y$ with outliers added, I want to compare the difference between $r_p$ and $\hat{r_p}$ with the difference between $r_k$ and $\hat{r_k}$

The problem is that I am not sure how to compare these results fairly. Kendall and Pearson, whilst both in the interval $[-1,1]$, tend to provide slightly different correlation estimates.

Let's say I find in one simulation that $r_p = 0.8$ and $r_k = 0.6$. And let's say that adding outliers causes a change of $0.2$ in both. I don't believe it is correct to conclude that their sensitivity is equal in that case. Seems to me that the "relative change" would be higher for $r_k$ in that particular instance. So this suggests some form of normalisation is needed to compare the two.

What is a correct, fair, way to perform this comparison?

  • 1
    $\begingroup$ It's a good question. But doesn't it have a trivial answer? Since the Kendall coefficients are based only on the ranks, and your "outliers" appear to be values with extreme ranks, then varying their values (provided they remain outlying) won't change the correlations at all. Only the Pearson correlation can change. $\endgroup$ – whuber Jun 13 '17 at 17:52
  • $\begingroup$ Agreed, the change in Kendall will be the same regardless of how far the outlier is (as long as it's further than any current observation). But nonetheless, if I'd like to compare this change to the corresponding change in Pearsons, how do I determine which is a larger change? (I'm aware that as a function of the outlier distance, Pearsons will be worse. But I'm interested in a comparison of individual cases also). Thanks :-) $\endgroup$ – Patrick Jun 13 '17 at 20:14
  • 1
    $\begingroup$ There doesn't seem to be any change to compare: once a point is an outlier, making it more of an outlier doesn't change Kendall's tau at all. The issue reduces to studying how the Pearson coefficient changes, which is relatively easy--and doesn't require comparing the two coefficients. $\endgroup$ – whuber Jun 13 '17 at 20:34
  • $\begingroup$ I see what you mean. But what about the initial change by adding the outliers in the first place? So forgetting about the "increasingly distant outlier" bit. If I simply say, ok, here is some data. Pearson is 0.8, Kendall is 0.6. I now add some outliers and find Pearson becomes 0.6 and Kendall becomes 0.4. How do I fairly say which one was affected more in that particular instance? $\endgroup$ – Patrick Jun 13 '17 at 20:51
  • $\begingroup$ That indeed is hard to say, because the coefficient measure different things. But at least we have arrived at a clearer formulation of your question! $\endgroup$ – whuber Jun 13 '17 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.