0
$\begingroup$

I have two datasets having values : s1 = [25 ,75, 25, 75, 25, 75, 25, 75, 25, 75] and s2 = [46.24, 73.16, 27.13, 74.30, 25, 72.52, 53.15, 75, 40.15, 69.86]

Pearson correlation coefficient = 0.90 and Spearman correlation coefficient = 0.90 (calculated using python library)

How correlation coefficient is 0.90, because for it to be 0.90 as per my understanding s1 and should be :
s1 = [25 ,75, 25, 75, 25, 75, 25, 75, 25, 75] and s2 = [20, 75, 25, 75, 25, 75, 25, 75, 25, 75]

ie. 9 datapoints should be similar and 1 different?

Is my understanding correct here ?

$\endgroup$
5
  • 3
    $\begingroup$ No that's not how correlation works. $\endgroup$ Mar 3, 2023 at 11:12
  • $\begingroup$ I think you should check the formula for correlation. It will give you a better idea about how it is calculated. $\endgroup$
    – T.E.G.
    Mar 3, 2023 at 11:15
  • $\begingroup$ Okay Thanks T.E.G $\endgroup$
    – Raj
    Mar 3, 2023 at 11:17
  • $\begingroup$ Can you please tell me what I am missing ? user2974951 $\endgroup$
    – Raj
    Mar 3, 2023 at 11:17
  • 2
    $\begingroup$ I suggest you start from en.wikipedia.org/wiki/Correlation and go through the formula. $\endgroup$ Mar 3, 2023 at 11:22

1 Answer 1

2
$\begingroup$

That is not what correlation means. Correlation, either Pearson or Spearman, can be $1$ when all of the values are different. Further, either correlation can take values below zero, so this “proportion of equal values” interpretation cannot apply to such a situation.

Pearson correlation is defined as the covariance between the two variables divided by the product of their standard deviations. Spearman correlation first transforms the variables to ranks (this number is the lowest, this next number is third-lowest, this next number is second-lowest, etc) and then applies Pearson correlation to the ranks. Applied to data, this equals:

$$ \rho(X,Y)=\dfrac{ \overset{n}{\underset{i=1}{\sum}} \left[ (X_i-\bar X)(Y_i-\bar Y) \right] }{ \sqrt{ \overset{n}{\underset{i=1}{\sum}}\left[ (X_i-\bar X)^2 \right) }\sqrt{ \overset{n}{\underset{i=1}{\sum}}\left[ (Y_i-\bar Y)^2 \right) } } $$

Take $X$ and $Y$ to be the raw values for Pearson correlation or the ranks for Spearman correlation.

An example of perfect Pearson correlation with no equal values is $X=(1,2,3)$, $Y=(11,21,31)$, which also gives perfect Spearman correlation. An example of perfect Spearman correlation, imperfect Pearson correlation, and no equal values is $X=(1,2,3)$, $Y=(11, 12, 15)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.