2
$\begingroup$

The formula for correlation coefficient is as follows:

$$\begin{align}\mathrm{corr} \left(\vec x, \vec y\right) = \frac{1}{n} \sum_{i=1}^n \frac{\left(x_i-\bar x\right)}{\sigma_x} \cdot \frac{\left(y_i - \bar y\right)}{\sigma_y} \end{align}$$

The idea behind correlation is the more similarly respective values in $\vec x$ and $\vec y$ deviates from their means, the closer $\mathbf{corr \left(\vec x, \vec y\right)}$ is to $\mathbf 1$. But how to read it off from this formula? It's not clear from the formula what it really calculates.

P.S. I don't know whether it's common on this site, but I've written the question to leave an answer. If it's not, let me know in comments.

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ I recommend your search around this site. This has been asked a ton of times and there are definitely answers available. $\endgroup$
    – Galen
    Oct 30, 2021 at 20:58
  • 1
    $\begingroup$ Please see stats.stackexchange.com/a/513608/919 for a deeper analysis of this issue. BTW, this is a strange formula for the correlation coefficient: usually the denominator is $n,$ not $n-1.$ With $n$ there, this clearly is the mean product of the standardized variables: that's straightforward to interpret. $\endgroup$
    – whuber ♦
    Oct 30, 2021 at 21:01
  • $\begingroup$ @Galen, Hello! I've already searched and haven't found anything similar. However, I can delete it, since I just wanted to share some intuition and don't actually need help. $\endgroup$
    – mathgeek
    Oct 30, 2021 at 21:04
  • $\begingroup$ @whuber, Hello! I've written $n-1$, since standard deviation is usually calculated with $n-1$ denominator. Thank you for the link. Should I delete the post? $\endgroup$
    – mathgeek
    Oct 30, 2021 at 21:06
  • 1
    $\begingroup$ You seem to be referring to an estimate of a standard deviation. The use of Greek variables $\sigma_x$ rather than latin variables $s_x$ implies you are referring to the SD itself rather than an estimate. $\endgroup$
    – whuber ♦
    Oct 30, 2021 at 21:08

1 Answer 1

Reset to default
4
$\begingroup$

Let's rewrite this formula as follows:

$$\begin{align} \mathrm{corr} \left(\vec x, \vec y\right) = \frac{\sum \left(x_i - \bar x\right)\left(y_i - \bar y\right)}{\sqrt{\sum\left(x_i - \bar x\right)^2}\sqrt{\sum\left(y_i - \bar y\right)^2}} = \\ = \frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|} \cdot \frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}} = \\ = \cos \theta \end{align}$$

Hence $\mathrm{corr} \left(\vec x, \vec y\right)$ is just $\cos \theta$, where $\theta$ is the angle between vectors $\left(\vec x - \bar x\right)$ and $\left(\vec y - \bar y\right)$.

When we ask:

Why similar deviations of respective values in $\vec x$ and $\vec y$ from their means (expressed in standard units) make correlation coefficient closer to $1$?

by that we actually ask:

Why being vector $\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$ closer to vector $\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$ makes the $\cos \theta$ closer to $1$?

since $\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$ and $\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$ are those deviations (divided by $n$).

The takeaway is that when we calculate $\mathrm{corr} \left(\vec x, \vec y\right)$ we actually calculate the cosine of the angle between unit vectors $\frac{\left(\vec x - \bar x\right)}{\|\vec x - \bar x\|}$ and $\frac{{\left(\vec y - \bar y\right)}}{{ \|\vec y - \bar y\|}}$. Hence this angle tend to $0$ as respective deviations of vectors $\vec x$ and $\vec y$ from their means (i.e. coordinates of the unit vectors) tend to be the same.

Improve this answer
$\endgroup$
7
  • 4
    $\begingroup$ You might enjoy reviewing the related posts that mention Anscombe's quartet. Although they refer to regression, they are relevant because the standardized least squares regression coefficient equals the correlation coefficient, thereby offering two more ways to interpret it (namely, by considering the regressions of $y$ vs $x$ and $x$ vs $y$). If you would like to explore this in more detail, my answer at stats.stackexchange.com/a/152034/919 provides software to produce more examples. $\endgroup$
    – whuber ♦
    Oct 30, 2021 at 21:22
  • $\begingroup$ sure, what doesn't look clearer on a n-dimensional space... $\endgroup$
    – carlo
    Oct 31, 2021 at 12:30
  • 2
    $\begingroup$ @carlo Only two vectors are involved. According to Euclid, they generate a two dimensional space. (It's irrelevant that these vectors are written down using $n$ numbers each.) Thus, it's possible to draw a simple picture of the vectors using just a sheet of paper or a computer screen. $\endgroup$
    – whuber ♦
    Oct 31, 2021 at 15:58
  • 1
    $\begingroup$ @carlo, And here is this picture (for 3D). The sheet of paper is actually the span of two given vectors. $\endgroup$
    – mathgeek
    Oct 31, 2021 at 16:48
  • 1
    $\begingroup$ @Carlo The formula lets us compute the angle. The Euclidean axioms let us see and conceptualize it. These advantages are available to any who would learn and practice the underlying concepts. $\endgroup$
    – whuber ♦
    Oct 31, 2021 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.