6
$\begingroup$

$$\rho = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)}$$

$\rho$ = Spearman's rank correlation coefficient
$d_i$ = difference between the two ranks of each observation
$n$ = number of observations

Given the Spearman's rank correlation above, it's clear to see the maximum is 1 as the smallest $d_i$ will be zero. I am trying to figure out why it is -1 by having two ranks in exactly reversed order.

I attach my calculation below, where the red rectangle represents the specific case of four rows, and I ordered two ranks in exactly reverse order. The purple rectangle is the actual calculation, and it indeed results in -1. I try to generalize in the green rectangle, but how does that formula result in 2? enter image description here

$\endgroup$
4
  • 2
    $\begingroup$ 1. Note that the formula you have there is a calculational formula mostly designed to speed up hand calculation; its not the definition, but an algebraically equivalent one derived from it. Spearman is just the correlation in the ranks and its minimum and maximum are the same as Pearson correlation (e.g. via Cauchy-Schwartz) because it is a Pearson, just calculated on the ranks. 2. However you can calculate it out easily enough - consider the rank differences when they're in exactly reversed order (when the correlation will be -1) and see that this formula gives -1 too $\endgroup$
    – Glen_b
    Feb 28 at 9:14
  • $\begingroup$ @Glen_b Thanks, I have put my calculation above, how does the green rectangle portion results in 2? $\endgroup$
    – william007
    Feb 28 at 9:55
  • $\begingroup$ The $2$ comes from having both $(1-m)^2$ and $(m-1)^2$, and then both $(3-m)^2$ and $(m-3)^2$, etc. in the sum. Just counting once, you would have a sum of $\frac16(m-1)m(m+1)$ $\endgroup$
    – Henry
    Feb 28 at 17:23
  • 3
    $\begingroup$ My understanding is that the formula above is only valid when there are no ties, otherwise you need to perform the calculation for Pearson's $r$ on ranks (including tied ranks). $\endgroup$
    – Alexis
    Feb 28 at 17:24

2 Answers 2

9
$\begingroup$

See Wikipedia for the definition. Note that Spearman correlation is just the usual Pearson correlation, but calculated using the ranks of the data, not the data itself.

So the reason it is always in the interval $[-1,1]$ is by the same proof as for the Pearson correlation. By using the Cauchy-Schwartz inequality.

$\endgroup$
5
  • 1
    $\begingroup$ +1 Because of the discrete nature of the ranks, it's in principle possible that Spearman's $\rho$ doesn't attain these endpoints. Thus, a complete answer would also point out (trivially) that when the ranks all agree, or when they agree upon reversal, both the values $1$ and $-1$ are possible. That proves $[-1,1]$ is gives the tightest bounds and helps emphasize that the crux of the matter is $|\rho|$ cannot *exceed$ $1.$ $\endgroup$
    – whuber
    Feb 28 at 13:38
  • 1
    $\begingroup$ @whuber: yes, but the same point arise for the usual Pearson correlation, where the bounds $\pm 1$ cannot be attained if the marginal distributions differ in shape (and not only in location/scale) $\endgroup$ Feb 28 at 14:21
  • 1
    $\begingroup$ That doesn't appear relevant, because the current thread is about data, not about theoretical distributions. $\endgroup$
    – whuber
    Feb 28 at 16:08
  • $\begingroup$ Thanks, but I want to find out why the calculation in green rectangle results in 2? $\endgroup$
    – william007
    Feb 28 at 16:29
  • $\begingroup$ @william007 Do you have an ties? The formula you presented is a shortcut which is, as I understand it, only valid when there are no ties. $\endgroup$
    – Alexis
    Feb 28 at 17:25
5
$\begingroup$

As you note the sum includes $1^2+3^2+5^2+\cdots+(m-1)^2$ or $\sum_1^{m/2} (2r-1)^2$. For simplicity I'll look at the case when there are an even number of rows, ie when $m$ is even.

You can expand the sum by expanding bracket as $$\sum_1^{m/2} (2r-1)^2 = 4\sum r^2-4\sum r +\sum 1$$ and use standard formulae for the sums: $\sum_1^n r^2 = \frac16 n(n+1)(2n+1)$ and $\sum_1^n r =\frac12 n(n+1)$

This gives $$\frac46 (m/2)((m/2)+1)(m+1) - \frac42 m/2(m/2+1) + m/2= \frac16 (m^3-m).$$

We can now substitute this into your expression: $$\frac{6\cdot(2\sum_1^{m/2} (2r-1)^2)}{m^3-m}$$ $$=\frac{6\cdot(2\cdot \frac16 (m^3-m)}{m^3-m}$$ $$=\frac{2 (m^3 -m)}{m^3-m}$$ $$=2$$

And so the Spearman rank coefficient is -1

So in this case, one can directly calculate the value. The standard formulae may not be familiar to you. Typically they are proved by induction, but a visual proof is offered on maths StackExchange: https://math.stackexchange.com/questions/122546/gaussian-proof-for-the-sum-of-squares

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.