Exact meaning of correlation coefficient

Question

What is the exact meaning of the entries of a correlation coefficients matrix?

I have spent time researching this, but could find only approximate interpretations which give me no good understanding of results. It's very clear to me what a correlation coefficient of 1, 0 or -1 mean, but I dislike the simplistic explanations I found for the numbers in between (e.g., 0-0.3 is scarce correlation, 0.3-0.7 is moderate, 0.7-1 is strong).

This gives no understanding about the difference between 0.99 and 0.85. As an engineer, I want to be able to calculate a threshold above which results fulfill requirements.

There is only one precise numerical interpretation I found in this page, but from the way it's stated I'm not sure if it's exact:

The value of r squared is typically taken as “the percent of variation in one variable explained by the other variable,” or “the percent of variation shared between the two variables.”

Is this correct? Are there other properties of the correlation coefficient one can use to analyse and explain the behaviour of a set of signals. Any mathematical insight related to this parameter would be appreciated.

Answer 1

Correlation has several exact meanings depending on how it's defined or derived. Some of them are listed in https://stats.stackexchange.com/a/104577/36229 These "13+2 Ways" are helpful here, because one of those ways might be more meaningful to you than another. There is great value in making connections between concepts that appear distinct at first.

The definition you highlighted about corresponds to number five, "The Square Root of the Ratio of Two Variances".

However, the definition that I was originally taught, and that I think is most intuitive to new stats students, is:

Correlation between $X$ and $Y$ is the strength of the linear relationship between $X$ and $Y$.

This also happens to correspond to the definition on Wikipedia, which also has some nice graphical demonstrations of correlation.

But you aren't asking about definitions as such. You want to know how to interpret intermediate correlation values, like 0.6. Unfortunately, correlation is a unitless index. It sacrifices interpretability for the ability to compare it freely across applications.

In keeping with the "linearity" theme, let's look at number three, the "standardized slope of the regression line." I won't repeat what is already written in the "13 Ways" article, but I want to draw your attention to Figure 2 of that paper, which shows that when you rescale a regression line by standardizing X and Y, the rescaled slope is actually restricted to the gray area in the figure. That is, there is a maximum possible slope for the rescaled line. Correlation, then, is just the extent to which this maximal slope would be achieved upon rescaling. So a correlation of 0.6 between X and Y implies that, upon standardization of X and Y, the rescaled slope of the line between them would be 60% of the maximal possible steepness admitted by the measurement scales of X and Y.

I happen to have recently published a blog post on a related topic (which ironically began its life as an answer on this site) that might help cement this relationship if you like working with equations: http://www.gregwerbin.com/blog/from-covariance-to-linear-regression