Why is the correlation coefficient a measure of the strength of linear relationship?

Question

I read some other topics on this but the concept still isn't clear to me. Given random variables $X$ and $Y$, I believe the correlation coefficient $\rho=\frac{Cov(X,Y)}{\sigma_X\sigma_Y}$ has an absolute value of 1 if and only if $Y$ is a linear function of $X$. I'm trying to get an intuitive understanding on why $\rho$ measures the strength of a specifically linear relationship and not something else? Is there a way to see this without involving parameters of least squares linear regression?

Answer 1

First You can see it by Using the Cauchy-Schwarz inequality. But I use another method.

Define

$$h(t)=E\bigg((X-\mu_x)t+(Y-\mu_y)\bigg)^2\geq 0$$

so

$$h(t)=t^2 Var(X)+2t cov(X,Y)+Var(Y)=at^2+bt+c$$

since $h(t)\geq 0 $ so $\Delta\leq 0$ in hence

$$\big(2 cov(X,Y)\big)^2-4Var(X) Var(Y)\leq 0$$ so $$cov^2(X,Y)\leq \sigma^2_x \sigma^2_y$$

Now

$$|\rho|=1$$

$$\Leftrightarrow$$ $$cov^2(X,Y)= \sigma^2_x \sigma^2_y$$ $$\Leftrightarrow$$ $$\Delta=0$$

$$\Leftrightarrow$$ $$h(t_1)=E\bigg((X-\mu_x)t_1+(Y-\mu_y)\bigg)^2=0$$ where $t_1=\frac{-b}{2a}$

$$\Leftrightarrow$$ $$P((X-\mu_x)t_1+(Y-\mu_y)=0)=1$$

$$\Leftrightarrow$$ almost surely $$Y=-t_1X+(\mu_y+t_1\mu_x)$$