3
$\begingroup$

What is the exact meaning of the entries of a correlation coefficients matrix?

I have spent time researching this, but could find only approximate interpretations which give me no good understanding of results. It's very clear to me what a correlation coefficient of 1, 0 or -1 mean, but I dislike the simplistic explanations I found for the numbers in between (e.g., 0-0.3 is scarce correlation, 0.3-0.7 is moderate, 0.7-1 is strong).

This gives no understanding about the difference between 0.99 and 0.85. As an engineer, I want to be able to calculate a threshold above which results fulfill requirements.

There is only one precise numerical interpretation I found in this page, but from the way it's stated I'm not sure if it's exact:

The value of r squared is typically taken as “the percent of variation in one variable explained by the other variable,” or “the percent of variation shared between the two variables.”

Is this correct? Are there other properties of the correlation coefficient one can use to analyse and explain the behaviour of a set of signals. Any mathematical insight related to this parameter would be appreciated.

$\endgroup$
  • 4
    $\begingroup$ Rodgers and Nicewander (1988) detailed 13 ways to look at the correlation coefficient, Rovine and Von Eye (1997) added another, and no doubt yet other perspectives could be added. Rodgers, J. L., and W. A. Nicewander. 1988. Thirteen ways to look at the correlation coefficient. American Statistician 42: 59–66. Rovine, M. J., and A. Von Eye. 1997. A 14th way to look at the correlation coefficient: Correlation as the proportion of matches. American Statistician 51: 42–46. $\endgroup$ – Nick Cox Sep 6 '16 at 14:47
  • 4
    $\begingroup$ The Rodgers & Nicewander article cited by @Nick is summarized in our thread at stats.stackexchange.com/questions/70969 along with a few more characterizations. $\endgroup$ – whuber Sep 7 '16 at 13:03
3
$\begingroup$

Correlation has several exact meanings depending on how it's defined or derived. Some of them are listed in https://stats.stackexchange.com/a/104577/36229 These "13+2 Ways" are helpful here, because one of those ways might be more meaningful to you than another. There is great value in making connections between concepts that appear distinct at first.

The definition you highlighted about corresponds to number five, "The Square Root of the Ratio of Two Variances".

However, the definition that I was originally taught, and that I think is most intuitive to new stats students, is:

Correlation between $X$ and $Y$ is the strength of the linear relationship between $X$ and $Y$.

This also happens to correspond to the definition on Wikipedia, which also has some nice graphical demonstrations of correlation.

But you aren't asking about definitions as such. You want to know how to interpret intermediate correlation values, like 0.6. Unfortunately, correlation is a unitless index. It sacrifices interpretability for the ability to compare it freely across applications.

In keeping with the "linearity" theme, let's look at number three, the "standardized slope of the regression line." I won't repeat what is already written in the "13 Ways" article, but I want to draw your attention to Figure 2 of that paper, which shows that when you rescale a regression line by standardizing X and Y, the rescaled slope is actually restricted to the gray area in the figure. That is, there is a maximum possible slope for the rescaled line. Correlation, then, is just the extent to which this maximal slope would be achieved upon rescaling. So a correlation of 0.6 between X and Y implies that, upon standardization of X and Y, the rescaled slope of the line between them would be 60% of the maximal possible steepness admitted by the measurement scales of X and Y.

I happen to have recently published a blog post on a related topic (which ironically began its life as an answer on this site) that might help cement this relationship if you like working with equations: http://www.gregwerbin.com/blog/from-covariance-to-linear-regression

$\endgroup$
  • $\begingroup$ Thanks for the answer, I just don't understand this sentence "Unfortunately, correlation is a unitless index. It sacrifices interpretability for the ability to compare it freely across applications." What you mean is that there is no absolute interpretation, but that relative interpretations are even easier, right? $\endgroup$ – raggot Sep 8 '16 at 9:01
  • $\begingroup$ @raggot yes that's what I mean $\endgroup$ – shadowtalker Sep 8 '16 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.