# How to think about the normalisation in correlation [closed]

So if we divide the covariance with the respective standard deviations we get the correlation which in turn is "measure of strength"

How does this divsion achive this property that covariance does not have?

• Let me get this straight: are you asking why we need to divide the covariance by the standard deviations and not remain contended with the mere covariance? Oct 30, 2022 at 4:51
• @User1865345 not exatcly, I am asking, why and in which way is the stranght displayed when we do the division, ofc it could be compared to not doing it! :)
– user291571
Oct 30, 2022 at 4:53
• "Strength" should not be impacted by the units you used. Note that if I change from calculating a covariance where one of the variables is in km to the same data but where it is measured in mm, will get a covariance whose numerical coefficient is a million times bigger (absent the units), while the amount of association didn't change one whit. We can remove the effect of the units by standardizing the variables $Z_x=(X-\mu_x)/\sigma_x$, which is now unitless, and looking at the covariance of those standardized variables; this will be a measure that remains unimpacted by a change of units. Oct 30, 2022 at 5:21
• @Glen_b thanks that explains a great deal.. the reduction by the mean is accounted for in the covariance tho, and does reduce effects of the magnutide, maybe there are two "magnitues" that we correct for, both size and variablity?
– user291571
Oct 30, 2022 at 5:27
• Yes, the mean correction is repeated in the covariance step, but saying "then compute $E(Z_x,Z_y)$" rather than "compute their covariance" would result in my comment less clearly conveying the benefit of removing the scale by the commonly-understood process of standardizing. We could just divide by $\sigma$ and then mean correct by the scaled mean in the covariance step, but again that would be less clearly motivated. Understanding a correlation as a covariance of standardized variables seems to me to be much the clearer step, for all that the mean correction part gets repeated. Oct 30, 2022 at 5:37

Both the covariance and (Pearson's) correlation quantify the strength of the linear relationship between two random variables. The convenience of using the product of the standard deviations to normalize the covariance (via the Cauchy-Schwarz inequality) is the boundedness (and removal of units as glen_b mentioned in the comments). The covariance could be any real number, but the correlation is bounded to [-1,1]. This makes comparing correlations more convenient than comparing covariances.

A confusing reversal can occur:

$$|\operatorname{Cov}[X,Y]| < |\operatorname{Cov}[U,V]|$$ while $$|\operatorname{Corr}[X,Y]| > |\operatorname{Corr}[U,V]|$$ simply due to the variables having sufficiently different scales. Correlation removes the effects of such scale, showing a certain form of scale invariance that the covariance does not have.

The covariance has homogeneity of scale while the standard deviations have absolute homogeneity of scale. Thus in the correlation you have positive invariance of scale (and negative scaling in one variable at at time will give a reflection). This kind of scale invariance achieves the removal of units that glen_b mentioned.

Here is a self-contained Python example:

from scipy.stats import pearsonr
import numpy as np

# Utility functions
abscorr = lambda x,y: np.abs(pearsonr(x,y))

def abscov(x,y):
return np.abs(np.mean((x - np.mean(x)) * (y - np.mean(y))))

# Data
np.random.seed(0)

x = np.random.normal(size=1000)
y = 3 * x + np.random.normal(size=1000)

u = np.random.normal(1,10**3,size=1000)
v = np.random.normal(1,10**3,size=1000)

# Results
print(abscov(x,y) < abscov(u,v))
print(abscorr(x,y) > abscorr(u,v))


which gives the result

True
True


In case it hints at the disparity, you can also plot such constructed cases (assuming the above code has been run):

import matplotlib.pyplot as plt

plt.scatter(x,y)
plt.xlabel('x')
plt.ylabel('y')
plt.show()

plt.scatter(u,v)
plt.xlabel('u')
plt.ylabel('v')
plt.show()


We can see the variables $$X$$ and $$Y$$ had a much smaller covariance than $$U$$ and $$V$$, but in a correlation sense their linear relationship is considerably stronger.

• Thanks, but there are two sets of "scales" tho, right? both magnitues and variablity, as I asked glen_B
– user291571
Oct 30, 2022 at 5:41
• @NoChance For normally-distributed variables, as in the above coded example, there is one scale parameter (the standard deviation) for each random variable. Are you referring the mean-centering as "magnitudes", or something else? Oct 30, 2022 at 5:42
• isnt the mean also a paramenter? I am not sure it is only boundedness that is the good part of correlation, but the homogenisation with respect to varaiablity as well.
– user291571
Oct 30, 2022 at 5:54
• @NoChance That's right, the mean is a location parameter. Oct 30, 2022 at 5:55
• @NoChance Sure, the covariance has homogeneity of scale while the standard deviations have absolute homogeneity of scale. Thus in the correlation you have positive invariance of scale (and negative scaling in one variable at at time will give a reflection). This kind of scale invariance achieves the removal of units that glen_b mentioned. Oct 30, 2022 at 5:59