5
$\begingroup$

What's the intuition of dividing by standard deviations in correlation?

The numerator is more intuitive as its supposed to measure "how much $X$ affects $Y$". But why does one have to divide with the standard deviations?

$\endgroup$
1
$\begingroup$

Dividing by the standard deviation normalizes the scale of the measure.

Let's say you are correlating distance to nearest grocery store with commute distance to work.

If we compute these correlations using distance in kilometers, and then repeat the analysis with distance in astronomical units (AU), do you think the correlation should change or stay the same? What if you express the distance to the grocery store in terms of meters instead?

If you think it should stay the same (I hope you do), then you need some way to adjust for the different scale of measures. Normalizing by standard deviations accomplishes this. What you are actually measuring with a correlation is "how much do changes in X occur along with changes in Y relative to how much X and Y each vary."

$\endgroup$
  • 1
    $\begingroup$ Umm do all "normalized" become the same magnitude sort of like when one normalizes to "unit vector length" in linear algebra? $\endgroup$ – mavavilj May 16 '18 at 17:37
  • 1
    $\begingroup$ The normalization due to standard deviation always leads to a value between -1 and 1. $\endgroup$ – Satwik Bhattamishra May 16 '18 at 17:42
1
$\begingroup$

The numerator is the covariance and thus without the denominator, the correlation coefficient (Pearson) is just the covariance between $X$ and $Y$. It is divided by the standard deviation to normalize the value and give us an interpretable strength. Covariance is a measure of how two variables change together but its magnitude is unbounded whereas standard deviation is a measure of dispersion of data. Dividing the covariance by the standard deviation gives us an interpretable value between -1 and 1.

$\endgroup$
0
$\begingroup$

The idea is to normalize in order to get a measure in $[-1,1]$ whatever the ranges of $X$ and $Y$. The aim being to be able to know the degree of linear dependency by looking at a single number.

To illustrate, let us take the example of two perfectly correlated variables $X$ and $Y$ such that $Var(X) = Var(Y) = 10$ (e.g. $X = Y$). In this case:

  • $cov(X,Y) = 10$ and $\rho(X,Y) = 1$.
  • $cov(5X,5Y) = 250$ but the correlation is still $1$: $\rho(5X,5Y) =1$

Looking simply at the covariance, it is impossible to compare the degree of linear dependency between the two couples of variables $(X,Y)$ and $(5X,5Y)$. All you can say is that both $(X,Y)$ and $(5X,5Y)$ move in the same direction.

The correlation on the other hand enables us to compare the degree of linear dependency between the $(X,Y)$ and $(5X,5Y)$ and conclude that the strength of linear dependency is the same.

The same argument can be applied for any value of correlation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.