Can anyone tell me why this near-perfect correlation yields a very low correlation coefficient?

a <- c(1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1)
b <- c(1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1)


Whereas this near-perfect correlation yields a very high coefficient

a <- c(1,2,3,4,5,6,8,8,9,10,11,12,13,14,15,16)
b <- c(1,2,3,4,5,6,7,8,9,10,11,13,13,14,15,16)


I am running a study testing the agreement between two binary variables where, like my first example, there is near-ubiquitous agreement between the two variables (122/128 agree), yet, like my first example, I am getting a very low correlation coefficient and a very high p-value for the test. Is there any solution? Am I going mad?

  • 3
    $\begingroup$ You assertion that the first set of data has "near perfect correlation" doesn't fit with the actual definition of the correlation coefficient in statistics. Can you explain why you think "often having the same value" implies "near perfect correlation"? $\endgroup$ – Glen_b May 31 '18 at 6:31
  • 1
    $\begingroup$ Correlation implies agreement/disagreement between fluctuations. If you are interested in the amount of overlap, you can use a distance measure like cosine similarity. $\endgroup$ – Moss Murderer May 31 '18 at 7:40
  • 1
    $\begingroup$ Correlation is really nothing like bivariate agreement. Take some random variable X that's got mean around 0 and sd. around 1. Let Y = 1000 + 10 X. Then X and Y are always very far apart, but they're perfectly correlated. $\endgroup$ – Glen_b May 31 '18 at 12:46
  • 2
    $\begingroup$ "how do I demonstrate agreement" is asking not for a test but a way to measure disagreement/agreement. There are many possible measures of how similar two things are, depending on circumstances, but you have specified too little about those circumstances to offer useful advice. What are you measuring? What values can these things take? $\endgroup$ – Glen_b May 31 '18 at 23:22
  • 1
    $\begingroup$ Sorry - I somehow completely missed the part right at the bottom of your question. With binary variables, would not the proportion of times they are equal be a reasonable measure of agreement? [You mentioned a test several times but it's not at all clear to me what the null and alternative could be] $\endgroup$ – Glen_b Jun 1 '18 at 8:06

A correlation coefficient measures the degree of variation that is shared between two variables, as a fraction of the total variance of both. In your first example, variables $a$ and $b$ are almost perfectly constant. The only variance that occurs is that one of their elements differs from the others, being higher by 1. However, since this occurs at different points in $a$ & $b$, the variance is not shared.

In other words, for a correlation coefficient to be high, you need it to be such that when $a$ goes up, $b$ goes up, and when $a$ goes down, $b$ goes down.

You seem to be interested in a different metric, which essentially tells you how close the two variables are to being equal across their paired observations. This is not what a correlation does. In fact, correlation coefficients are not sensitive to the mean or scale of either variable. E.g. in your second example, if you added 1,000 to the values of $b$, you would still get the same correlation out, even though the difference in values between $a$ & $b$ would then be very high.

| cite | improve this answer | |
  • $\begingroup$ Thank you @Ruben van Bergen. So if, for example, my hypothesis was that if a person has a nose (no nose = 0, nose =1) they tend to have two nostrils (other than two nostrils = 0, two nostrils = 1), how do I confirm this with a statistical test on a sample of humans, for whom the two variables will almost always be (1,1) for every observation? $\endgroup$ – llewmills Jun 1 '18 at 0:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.