# What does correlation formula really tell you?

The formula for correlation coefficient is as follows:

\begin{align}\mathrm{corr} \left(\vec x, \vec y\right) = \frac{1}{n} \sum_{i=1}^n \frac{\left(x_i-\bar x\right)}{\sigma_x} \cdot \frac{\left(y_i - \bar y\right)}{\sigma_y} \end{align}

The idea behind correlation is the more similarly respective values in $$\vec x$$ and $$\vec y$$ deviates from their means, the closer $$\mathbf{corr \left(\vec x, \vec y\right)}$$ is to $$\mathbf 1$$. But how to read it off from this formula? It's not clear from the formula what it really calculates.

P.S. I don't know whether it's common on this site, but I've written the question to leave an answer. If it's not, let me know in comments.

• I recommend your search around this site. This has been asked a ton of times and there are definitely answers available. Commented Oct 30, 2021 at 20:58
• Please see stats.stackexchange.com/a/513608/919 for a deeper analysis of this issue. BTW, this is a strange formula for the correlation coefficient: usually the denominator is $n,$ not $n-1.$ With $n$ there, this clearly is the mean product of the standardized variables: that's straightforward to interpret.
– whuber
Commented Oct 30, 2021 at 21:01
• @Galen, Hello! I've already searched and haven't found anything similar. However, I can delete it, since I just wanted to share some intuition and don't actually need help. Commented Oct 30, 2021 at 21:04
• @whuber, Hello! I've written $n-1$, since standard deviation is usually calculated with $n-1$ denominator. Thank you for the link. Should I delete the post? Commented Oct 30, 2021 at 21:06
• You seem to be referring to an estimate of a standard deviation. The use of Greek variables $\sigma_x$ rather than latin variables $s_x$ implies you are referring to the SD itself rather than an estimate.
– whuber
Commented Oct 30, 2021 at 21:08

Let's rewrite this formula as follows:

\begin{align} \mathrm{corr} \left(\vec x, \vec y\right) = \frac{\sum \left(x_i - \bar x\right)\left(y_i - \bar y\right)}{\sqrt{\sum\left(x_i - \bar x\right)^2}\sqrt{\sum\left(y_i - \bar y\right)^2}} = \\ = \frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|} \cdot \frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}} = \\ = \cos \theta \end{align}

Hence $$\mathrm{corr} \left(\vec x, \vec y\right)$$ is just $$\cos \theta$$, where $$\theta$$ is the angle between vectors $$\left(\vec x - \bar x\right)$$ and $$\left(\vec y - \bar y\right)$$.

Why similar deviations of respective values in $$\vec x$$ and $$\vec y$$ from their means (expressed in standard units) make correlation coefficient closer to $$1$$?

Why being vector $$\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$$ closer to vector $$\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$$ makes the $$\cos \theta$$ closer to $$1$$?

since $$\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$$ and $$\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$$ are those deviations (divided by $$n$$).

The takeaway is that when we calculate $$\mathrm{corr} \left(\vec x, \vec y\right)$$ we actually calculate the cosine of the angle between unit vectors $$\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$$ and $$\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$$. Hence this angle tend to $$0$$ as respective deviations of vectors $$\vec x$$ and $$\vec y$$ from their means (i.e. coordinates of the unit vectors) tend to be the same.

• You might enjoy reviewing the related posts that mention Anscombe's quartet. Although they refer to regression, they are relevant because the standardized least squares regression coefficient equals the correlation coefficient, thereby offering two more ways to interpret it (namely, by considering the regressions of $y$ vs $x$ and $x$ vs $y$). If you would like to explore this in more detail, my answer at stats.stackexchange.com/a/152034/919 provides software to produce more examples.
– whuber
Commented Oct 30, 2021 at 21:22
• sure, what doesn't look clearer on a n-dimensional space... Commented Oct 31, 2021 at 12:30
• @carlo Only two vectors are involved. According to Euclid, they generate a two dimensional space. (It's irrelevant that these vectors are written down using $n$ numbers each.) Thus, it's possible to draw a simple picture of the vectors using just a sheet of paper or a computer screen.
– whuber
Commented Oct 31, 2021 at 15:58
• @carlo, And here is this picture (for 3D). The sheet of paper is actually the span of two given vectors. Commented Oct 31, 2021 at 16:48
• @Carlo The formula lets us compute the angle. The Euclidean axioms let us see and conceptualize it. These advantages are available to any who would learn and practice the underlying concepts.
– whuber
Commented Oct 31, 2021 at 17:29