How to think about the normalisation in correlation [closed]

Question

So if we divide the covariance with the respective standard deviations we get the correlation which in turn is "measure of strength"

How does this divsion achive this property that covariance does not have?

Answer 1

Both the covariance and (Pearson's) correlation quantify the strength of the linear relationship between two random variables. The convenience of using the product of the standard deviations to normalize the covariance (via the Cauchy-Schwarz inequality) is the boundedness (and removal of units as glen_b mentioned in the comments). The covariance could be any real number, but the correlation is bounded to [-1,1]. This makes comparing correlations more convenient than comparing covariances.

A confusing reversal can occur:

$$|\operatorname{Cov}[X,Y]| < |\operatorname{Cov}[U,V]|$$ while $$|\operatorname{Corr}[X,Y]| > |\operatorname{Corr}[U,V]|$$ simply due to the variables having sufficiently different scales. Correlation removes the effects of such scale, showing a certain form of scale invariance that the covariance does not have.

The covariance has homogeneity of scale while the standard deviations have absolute homogeneity of scale. Thus in the correlation you have positive invariance of scale (and negative scaling in one variable at at time will give a reflection). This kind of scale invariance achieves the removal of units that glen_b mentioned.

---

Here is a self-contained Python example:

from scipy.stats import pearsonr
import numpy as np

# Utility functions
abscorr = lambda x,y: np.abs(pearsonr(x,y)[0])

def abscov(x,y):
    return np.abs(np.mean((x - np.mean(x)) * (y - np.mean(y))))

# Data 
np.random.seed(0)

x = np.random.normal(size=1000)
y = 3 * x + np.random.normal(size=1000)

u = np.random.normal(1,10**3,size=1000)
v = np.random.normal(1,10**3,size=1000)

# Results
print(abscov(x,y) < abscov(u,v))
print(abscorr(x,y) > abscorr(u,v))

which gives the result

True
True

---

In case it hints at the disparity, you can also plot such constructed cases (assuming the above code has been run):

import matplotlib.pyplot as plt

plt.scatter(x,y)
plt.xlabel('x')
plt.ylabel('y')
plt.show()

plt.scatter(u,v)
plt.xlabel('u')
plt.ylabel('v')
plt.show()

(X,Y)	(U,V)

We can see the variables $X$ and $Y$ had a much smaller covariance than $U$ and $V$, but in a correlation sense their linear relationship is considerably stronger.